2024-04-18T22:08:49.479 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-18T22:08:49.479 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --description powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/ubuntu_latest tasks/admin_socket_objecter_requests thrashosds-health} --name teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815 -- /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/orig.config.yaml 2024-04-18T22:08:49.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T22:08:49.598 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815 branch: squid description: powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/ubuntu_latest tasks/admin_socket_objecter_requests thrashosds-health} email: ceph-qa@ceph.com first_in_suite: false job_id: '7662815' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client.0: admin socket: /var/run/ceph/ceph-$name.asok mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MDS_TRIM\) - \(MDS_SLOW_REQUEST\) - MDS_SLOW_METADATA_IO - Behind on trimming - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d0bf63dd179c05751e4be49477389aa290c73089 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - client.0 - - osd.0 - - osd.1 - - osd.2 seed: 3123 sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 sleep_before_teardown: 0 subset: 2/4 suite: powercycle suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d0bf63dd179c05751e4be49477389aa290c73089 targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbH8GKYerZQQDCKKdwmL/B1tET5gfWTb3XjPHYxUmrJk8gDxj5gUtY+wVDry20xQrTUsW1x9Uy5v8409hqsAB0= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCvJJPlGl728mrthu3bgEjtyqIpgscw+UcRys7a+eQDk1cwxArsTUymySL7Io/44TMwx+MtsJzo+B9p3Td7zBGo= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhdpWrsge4ltjd4TTC93pKKwl5G/UHxfWPnieTgbSYJPGMI3suRRBaQ0BE0cIhdir8CUR9GZxCXRXL528v9Epo= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKv3SFUyG8tl2FjpHsPmjOvqm7BKPU1qsnd/gKJtDtiPNZKhyVtpZYEu5R5ykBHXxtVnbrV/+dNMFlPKQvvUXKU= tasks: - install: extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - thrashosds: chance_down: 1.0 powercycle: true timeout: 600 - radosbench: clients: - client.0 time: 60 - admin_socket: client.0: objecter_requests: test: http://git.ceph.com/?p={repo};a=blob_plain;f=src/test/admin_socket/objecter_requests;hb={branch} teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-18_21:32:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-18T22:08:49.598 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa; will attempt to use it 2024-04-18T22:08:49.599 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa/tasks 2024-04-18T22:08:49.599 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-18T22:08:49.600 INFO:teuthology.task.internal:Checking packages... 2024-04-18T22:08:49.623 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1a6a3e742fd81c6d70be70552499bff767652c32' 2024-04-18T22:08:49.623 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-18T22:08:49.623 INFO:teuthology.packaging:ref: None 2024-04-18T22:08:49.623 INFO:teuthology.packaging:tag: None 2024-04-18T22:08:49.623 INFO:teuthology.packaging:branch: squid 2024-04-18T22:08:49.623 INFO:teuthology.packaging:sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:08:49.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-18T22:08:49.918 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:08:49.919 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-18T22:08:49.925 INFO:teuthology.task.internal:no buildpackages task found 2024-04-18T22:08:49.925 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-18T22:08:50.006 INFO:teuthology.task.internal:Saving configuration 2024-04-18T22:08:50.019 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-18T22:08:50.033 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-18T22:08:50.058 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:01:25.351155', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKeDQlai05S45+BFYJKxA4psU2Kn6u+Ry9dCNiMzZwXb'} 2024-04-18T22:08:50.079 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:01:25.348836', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF0DIag+BGXwETbSOVlyvEHvbLxgZi0tl7jFCXBMSxoD'} 2024-04-18T22:08:50.099 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:01:25.347384', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE/8WJWbQ6lfT1HyDKgkOMo3DE2qTzPjGfG+NELlqvnV'} 2024-04-18T22:08:50.121 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:01:25.350002', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDH7mdC4mcIb81mFvLgkXEDlUPDRMQGoxgIMokUFaEsk'} 2024-04-18T22:08:50.121 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-18T22:08:50.127 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'] 2024-04-18T22:08:50.128 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['osd.0'] 2024-04-18T22:08:50.128 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['osd.1'] 2024-04-18T22:08:50.128 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['osd.2'] 2024-04-18T22:08:50.128 INFO:teuthology.run_tasks:Running task console_log... 2024-04-18T22:08:50.301 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f25cf68f280>, signals=[15]) 2024-04-18T22:08:50.301 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-18T22:08:50.307 INFO:teuthology.task.internal:Opening connections... 2024-04-18T22:08:50.308 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-04-18T22:08:50.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:08:50.389 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-18T22:08:50.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:08:50.467 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:08:50.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:08:50.545 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-18T22:08:50.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:08:50.627 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-18T22:08:50.634 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-04-18T22:08:50.640 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-04-18T22:08:50.640 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-04-18T22:08:50.685 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:08:50.685 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2024-04-18T22:08:50.685 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="22.04" 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=jammy 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:08:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:08:50.687 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-04-18T22:08:50.717 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-18T22:08:50.723 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-18T22:08:50.723 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="22.04" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=jammy 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:08:50.772 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:08:50.773 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-18T22:08:50.792 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-04-18T22:08:50.798 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-04-18T22:08:50.798 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="22.04" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=jammy 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:08:50.847 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:08:50.848 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:08:50.848 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-04-18T22:08:50.867 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-18T22:08:50.872 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-18T22:08:50.873 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Ubuntu" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="22.04" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_CODENAME=jammy 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:ID=ubuntu 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE=debian 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:08:50.919 INFO:teuthology.orchestra.run.smithi196.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:08:50.920 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-18T22:08:50.942 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-18T22:08:50.950 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-18T22:08:51.030 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-18T22:08:51.030 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:08:51.032 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:08:51.035 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:08:51.037 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:08:51.042 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-18T22:08:51.105 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-18T22:08:51.105 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:08:51.108 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:08:51.110 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:08:51.112 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:08:51.140 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-18T22:08:51.242 INFO:teuthology.run_tasks:Running task kernel... 2024-04-18T22:08:51.256 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-18T22:08:51.256 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'osd.0': {'kdb': 1, 'sha1': 'distro'}, 'osd.1': {'kdb': 1, 'sha1': 'distro'}, 'osd.2': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-18T22:08:51.257 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:08:51.257 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:08:51.258 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:08:51.258 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:08:51.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:08:51.263 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-18T22:08:51.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:08:51.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:08:51.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:08:51.265 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-18T22:08:51.266 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-18T22:08:51.266 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-18T22:08:51.310 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-56-generic 2024-04-18T22:08:51.310 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.15.0-56-generic 2024-04-18T22:08:51.311 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-04-18T22:08:51.313 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-56-generic 2024-04-18T22:08:51.313 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-56-generic 2024-04-18T22:08:51.313 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.15.0-56-generic 2024-04-18T22:08:51.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get clean 2024-04-18T22:08:51.313 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.15.0-56-generic 2024-04-18T22:08:51.314 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2024-04-18T22:08:51.315 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-56-generic 2024-04-18T22:08:51.315 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.15.0-56-generic 2024-04-18T22:08:51.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2024-04-18T22:08:51.448 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-18T22:08:51.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-04-18T22:08:51.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2024-04-18T22:08:51.459 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-04-18T22:08:51.601 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:08:51.621 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:08:51.621 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:08:51.621 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:08:51.627 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:08:51.638 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:08:51.699 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:08:51.705 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:08:51.709 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:08:51.728 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:08:51.782 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:08:51.786 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:08:51.810 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:08:51.864 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-18T22:08:51.883 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:08:51.883 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-18T22:08:51.896 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:08:51.943 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-18T22:08:52.063 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-18T22:08:52.079 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-18T22:08:52.143 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:08:52.149 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:08:52.177 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:08:52.255 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:08:52.257 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:08:52.280 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-18T22:08:52.286 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-18T22:08:52.303 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-18T22:08:52.309 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-18T22:08:52.310 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-18T22:08:52.315 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-18T22:08:52.315 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-18T22:08:52.373 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-18T22:08:52.373 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:08:52.373 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:08:52.384 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:08:52.391 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:08:52.395 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:08:52.395 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:08:52.397 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:08:52.421 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-18T22:08:52.443 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:08:52.444 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:08:52.466 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:08:52.467 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:08:52.470 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,754 kB] 2024-04-18T22:08:52.484 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-18T22:08:52.513 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-18T22:08:52.524 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:08:52.525 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:08:52.587 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-18T22:08:52.610 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.610 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:08:52.626 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-18T22:08:52.633 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-18T22:08:52.647 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:08:52.659 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:08:52.659 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,754 kB] 2024-04-18T22:08:52.662 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-18T22:08:52.666 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:08:52.666 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:08:52.667 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,754 kB] 2024-04-18T22:08:52.677 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:08:52.678 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:08:52.679 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,754 kB] 2024-04-18T22:08:52.687 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:08:52.704 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:08:52.705 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [295 kB] 2024-04-18T22:08:52.710 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [295 kB] 2024-04-18T22:08:52.712 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.715 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-18T22:08:52.718 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.727 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:08:52.730 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:08:52.733 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.734 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-18T22:08:52.740 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [295 kB] 2024-04-18T22:08:52.747 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.758 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [295 kB] 2024-04-18T22:08:52.760 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.760 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:08:52.768 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.769 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:08:52.796 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:08:52.801 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:08:52.801 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:08:52.808 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-18T22:08:52.811 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:08:52.811 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:08:52.812 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-18T22:08:52.812 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:08:52.812 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:08:52.813 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:08:52.814 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:08:52.816 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-18T22:08:52.821 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-04-18T22:08:52.832 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-18T22:08:52.840 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-18T22:08:52.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:08:52.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:08:52.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:08:52.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:08:52.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:08:52.877 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:08:52.882 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-04-18T22:08:52.895 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:08:52.896 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:08:52.896 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:08:52.897 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:08:52.898 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:08:52.911 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-04-18T22:08:52.917 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-18T22:08:52.920 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:08:52.921 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:08:52.922 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:08:52.922 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:08:52.922 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:08:52.936 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-18T22:08:52.941 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:08:52.942 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:08:52.943 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:08:52.944 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:08:52.945 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:08:52.945 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:08:52.947 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:08:52.948 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:08:52.999 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:08:53.056 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:08:53.056 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:08:53.059 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:08:53.061 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:08:53.452 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-04-18T22:08:53.517 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:08:53.545 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:08:53.554 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-18T22:08:53.564 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-18T22:08:53.567 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:08:53.569 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:08:53.569 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:08:53.571 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:08:53.572 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:08:53.573 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:08:53.582 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-18T22:08:53.583 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:08:53.585 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:08:53.586 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:08:53.588 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:08:53.589 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:08:53.590 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:08:53.594 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:08:53.595 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:08:53.596 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:08:53.597 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:08:53.598 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:08:53.599 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:08:53.599 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:08:53.599 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:08:53.600 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:08:53.602 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:08:53.627 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:08:53.629 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:08:53.767 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:08:53.767 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:08:53.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:08:53.846 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:08:53.849 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:08:53.877 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:08:53.930 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:08:53.932 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:08:53.966 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:08:54.012 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:08:54.012 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:08:54.042 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:08:54.043 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:08:54.046 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:08:54.078 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:08:58.027 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 12.9 MB in 2s (6,056 kB/s) 2024-04-18T22:08:58.364 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 12.9 MB in 3s (5,134 kB/s) 2024-04-18T22:08:58.397 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 12.9 MB in 3s (5,105 kB/s) 2024-04-18T22:08:58.996 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 12.9 MB in 3s (5,084 kB/s) 2024-04-18T22:08:59.395 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:08:59.413 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:08:59.472 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:08:59.653 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-18T22:08:59.653 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-18T22:08:59.661 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:08:59.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:08:59.728 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:08:59.776 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:08:59.794 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:08:59.806 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:08:59.806 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:08:59.806 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:08:59.807 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:08:59.807 INFO:teuthology.orchestra.run.smithi196.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:08:59.807 INFO:teuthology.orchestra.run.smithi196.stdout: python2.7-minimal 2024-04-18T22:08:59.807 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:08:59.808 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-18T22:08:59.808 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:08:59.808 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-18T22:08:59.808 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-18T22:08:59.809 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-18T22:08:59.810 INFO:teuthology.orchestra.run.smithi196.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:08:59.810 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-04-18T22:08:59.810 INFO:teuthology.orchestra.run.smithi196.stdout: thermald 2024-04-18T22:08:59.855 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:08:59.856 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-18T22:08:59.856 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:08:59.856 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-18T22:08:59.856 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-18T22:08:59.857 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-04-18T22:08:59.858 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:08:59.903 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-18T22:08:59.904 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-18T22:08:59.977 INFO:teuthology.orchestra.run.smithi196.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:08:59.977 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 113 MB of archives. 2024-04-18T22:08:59.977 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:08:59.977 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-18T22:09:00.037 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-04-18T22:09:00.037 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-04-18T22:09:00.054 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:00.054 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:00.054 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:00.055 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:00.055 INFO:teuthology.orchestra.run.smithi086.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:09:00.055 INFO:teuthology.orchestra.run.smithi086.stdout: python2.7-minimal 2024-04-18T22:09:00.055 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:00.056 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-18T22:09:00.056 INFO:teuthology.orchestra.run.smithi086.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:09:00.056 INFO:teuthology.orchestra.run.smithi086.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-18T22:09:00.057 INFO:teuthology.orchestra.run.smithi086.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-18T22:09:00.058 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-18T22:09:00.058 INFO:teuthology.orchestra.run.smithi086.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:09:00.058 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2024-04-18T22:09:00.058 INFO:teuthology.orchestra.run.smithi086.stdout: thermald 2024-04-18T22:09:00.105 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-18T22:09:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:09:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-18T22:09:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-18T22:09:00.106 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2024-04-18T22:09:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:09:00.192 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:09:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: python2.7-minimal 2024-04-18T22:09:00.193 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:00.194 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-04-18T22:09:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:09:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-18T22:09:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-18T22:09:00.196 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-04-18T22:09:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:09:00.196 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-04-18T22:09:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: thermald 2024-04-18T22:09:00.205 INFO:teuthology.orchestra.run.smithi086.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:00.206 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 113 MB of archives. 2024-04-18T22:09:00.206 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:09:00.206 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-18T22:09:00.241 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-04-18T22:09:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:09:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-18T22:09:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-18T22:09:00.243 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-04-18T22:09:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:09:00.342 INFO:teuthology.orchestra.run.smithi096.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:00.342 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 113 MB of archives. 2024-04-18T22:09:00.342 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:09:00.342 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-18T22:09:00.552 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:09:00.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:09:00.625 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:09:00.828 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-04-18T22:09:00.829 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-04-18T22:09:00.834 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-18T22:09:01.005 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:01.005 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:01.005 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:01.005 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:01.005 INFO:teuthology.orchestra.run.smithi032.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:09:01.005 INFO:teuthology.orchestra.run.smithi032.stdout: python2.7-minimal 2024-04-18T22:09:01.006 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:01.006 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-04-18T22:09:01.006 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:09:01.006 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-18T22:09:01.007 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-18T22:09:01.008 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-04-18T22:09:01.008 INFO:teuthology.orchestra.run.smithi032.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:09:01.008 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-04-18T22:09:01.008 INFO:teuthology.orchestra.run.smithi032.stdout: thermald 2024-04-18T22:09:01.066 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-04-18T22:09:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-18T22:09:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-18T22:09:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-18T22:09:01.066 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2024-04-18T22:09:01.066 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:09:01.215 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-18T22:09:01.244 INFO:teuthology.orchestra.run.smithi032.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:01.244 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 113 MB of archives. 2024-04-18T22:09:01.244 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:09:01.244 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-18T22:09:01.563 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-18T22:09:01.633 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-18T22:09:01.991 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-18T22:09:02.111 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-18T22:09:02.638 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-18T22:09:02.980 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-18T22:09:03.564 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-18T22:09:03.564 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-18T22:09:03.564 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-18T22:09:04.125 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-18T22:09:04.347 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-18T22:09:04.739 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-18T22:09:04.739 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-18T22:09:04.739 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-18T22:09:04.740 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-18T22:09:04.748 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 113 MB in 4s (25.3 MB/s) 2024-04-18T22:09:04.773 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-18T22:09:04.785 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-18T22:09:04.924 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-18T22:09:05.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-18T22:09:05.104 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-18T22:09:05.104 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-18T22:09:05.105 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-18T22:09:05.168 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-18T22:09:05.258 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-18T22:09:05.412 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-18T22:09:05.567 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 113 MB in 5s (23.0 MB/s) 2024-04-18T22:09:05.777 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-18T22:09:05.807 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 113 MB in 5s (21.4 MB/s) 2024-04-18T22:09:05.846 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-18T22:09:05.972 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-18T22:09:05.981 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-18T22:09:06.403 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 113 MB in 5s (23.1 MB/s) 2024-04-18T22:09:06.609 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-18T22:09:06.911 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:09:06.915 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:07.014 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:07.793 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:09:07.798 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:07.892 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:07.973 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:09:07.978 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:08.062 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:09:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:08.766 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:09.147 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-18T22:09:09.168 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:09.249 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:09.734 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-18T22:09:09.754 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:09.792 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:10.010 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-18T22:09:10.032 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:10.096 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-18T22:09:10.118 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:10.123 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:10.201 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:10.597 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-18T22:09:10.616 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-18T22:09:10.618 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:10.636 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:10.655 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:10.676 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:11.010 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-18T22:09:11.032 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:11.128 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:11.638 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-18T22:09:11.659 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:11.696 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:16.609 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:16.769 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:17.144 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:17.281 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:17.351 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:17.475 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:17.567 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-18T22:09:17.590 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-18T22:09:17.634 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:17.736 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:17.852 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:17.946 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-18T22:09:18.234 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-18T22:09:18.290 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:18.356 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:18.659 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-18T22:09:18.670 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-18T22:09:18.734 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:18.836 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:18.962 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:19.288 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:19.391 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:19.711 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-18T22:09:19.725 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-18T22:09:19.769 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:23.251 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-18T22:09:23.288 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:23.326 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:23.677 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-18T22:09:23.713 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:23.751 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:24.485 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-18T22:09:24.522 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:24.560 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:25.782 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-18T22:09:25.820 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-18T22:09:25.878 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:26.032 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:26.154 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:26.352 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:26.478 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:26.521 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:26.626 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:26.752 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-18T22:09:26.811 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:26.932 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:27.072 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:27.267 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-18T22:09:27.963 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:28.104 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:28.462 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:28.584 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:28.710 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-18T22:09:28.814 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:29.149 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:29.269 INFO:teuthology.orchestra.run.smithi086.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:29.269 INFO:teuthology.orchestra.run.smithi086.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:29.269 INFO:teuthology.orchestra.run.smithi086.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:29.269 INFO:teuthology.orchestra.run.smithi086.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:29.464 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:30.157 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:30.158 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:30.158 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:30.158 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:30.185 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-18T22:09:30.310 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-18T22:09:30.349 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:30.569 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:30.685 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-18T22:09:30.822 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:30.857 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:30.949 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-18T22:09:31.094 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:31.725 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:31.979 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-18T22:09:32.122 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-18T22:09:32.248 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:32.301 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:09:32.301 INFO:teuthology.orchestra.run.smithi196.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:32.308 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-18T22:09:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:32.433 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-18T22:09:32.585 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:32.604 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:32.640 INFO:teuthology.orchestra.run.smithi086.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:09:32.640 INFO:teuthology.orchestra.run.smithi086.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:33.727 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-18T22:09:33.836 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-18T22:09:33.987 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:34.044 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:09:34.045 INFO:teuthology.orchestra.run.smithi096.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:34.614 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:36.074 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-18T22:09:36.174 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-18T22:09:36.301 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-18T22:09:36.396 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:09:36.396 INFO:teuthology.orchestra.run.smithi032.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:46.351 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:09:46.351 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:09:46.379 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:46.393 INFO:teuthology.orchestra.run.smithi196.stdout:Generating grub configuration file ... 2024-04-18T22:09:46.410 INFO:teuthology.orchestra.run.smithi086.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:09:46.410 INFO:teuthology.orchestra.run.smithi086.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:09:46.428 INFO:teuthology.orchestra.run.smithi086.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:46.456 INFO:teuthology.orchestra.run.smithi086.stdout:Generating grub configuration file ... 2024-04-18T22:09:46.673 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:46.687 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:46.720 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:46.741 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:46.968 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:46.973 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:47.019 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:47.025 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:47.048 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:47.098 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:47.104 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:47.250 INFO:teuthology.orchestra.run.smithi086.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:47.251 INFO:teuthology.orchestra.run.smithi086.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:47.251 INFO:teuthology.orchestra.run.smithi086.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:47.265 INFO:teuthology.orchestra.run.smithi196.stdout:done 2024-04-18T22:09:47.287 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-04-18T22:09:47.735 DEBUG:teuthology.orchestra.run.smithi086:> dpkg -s linux-image-generic 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Package: linux-image-generic 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Status: install ok installed 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Priority: optional 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Section: kernel 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Installed-Size: 21 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: amd64 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Source: linux-meta 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:47.760 INFO:teuthology.orchestra.run.smithi086.stdout:Recommends: thermald 2024-04-18T22:09:47.760 INFO:teuthology.orchestra.run.smithi086.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:47.760 INFO:teuthology.orchestra.run.smithi086.stdout: available. 2024-04-18T22:09:47.760 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-18T22:09:47.760 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:09:47.760 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:09:47.760 INFO:teuthology.task.kernel:Installing distro kernel on osd.0... 2024-04-18T22:09:47.760 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:09:47.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get clean 2024-04-18T22:09:47.765 DEBUG:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2024-04-18T22:09:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2024-04-18T22:09:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2024-04-18T22:09:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2024-04-18T22:09:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2024-04-18T22:09:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 21 2024-04-18T22:09:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:47.784 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2024-04-18T22:09:47.784 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-18T22:09:47.784 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:09:47.785 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:09:47.785 INFO:teuthology.task.kernel:Installing distro kernel on osd.2... 2024-04-18T22:09:47.785 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:09:47.785 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2024-04-18T22:09:47.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2024-04-18T22:09:48.006 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-18T22:09:48.092 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:09:48.105 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:09:48.126 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:09:48.178 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:09:48.178 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:09:48.178 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:09:48.184 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:09:48.206 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:09:48.238 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:09:48.238 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:09:48.265 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:48.292 INFO:teuthology.orchestra.run.smithi096.stdout:Generating grub configuration file ... 2024-04-18T22:09:48.682 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:48.699 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:49.121 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:49.127 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:49.194 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:49.199 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:49.378 INFO:teuthology.orchestra.run.smithi096.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:49.402 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:09:49.416 INFO:teuthology.orchestra.run.smithi096.stdout:done 2024-04-18T22:09:49.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:09:49.472 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:09:49.480 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:09:49.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:09:49.559 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:09:49.652 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-18T22:09:49.653 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-18T22:09:49.760 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-18T22:09:49.761 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-18T22:09:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:09:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:49.847 DEBUG:teuthology.orchestra.run.smithi086:> dpkg -s linux-image-generic 2024-04-18T22:09:49.865 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Package: linux-image-generic 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Status: install ok installed 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Priority: optional 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Section: kernel 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Installed-Size: 21 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: amd64 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Source: linux-meta 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:Recommends: thermald 2024-04-18T22:09:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:49.906 INFO:teuthology.orchestra.run.smithi086.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:49.906 INFO:teuthology.orchestra.run.smithi086.stdout: available. 2024-04-18T22:09:49.906 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:49.926 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:49.927 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-04-18T22:09:49.927 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-04-18T22:09:49.927 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-18T22:09:49.927 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:09:49.927 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:09:49.927 INFO:teuthology.task.kernel:Installing distro kernel on osd.1... 2024-04-18T22:09:49.927 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:09:49.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-04-18T22:09:49.935 INFO:teuthology.orchestra.run.smithi196.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-18T22:09:49.935 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:49.936 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:49.936 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:49.936 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:49.936 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:09:49.936 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:09:49.936 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:49.954 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.E3LuXkWrKO 2024-04-18T22:09:49.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub/grub.cfg /tmp/tmp.E3LuXkWrKO 2024-04-18T22:09:49.980 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:49.982 DEBUG:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2024-04-18T22:09:50.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.E3LuXkWrKO 2024-04-18T22:09:50.037 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 21 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2024-04-18T22:09:50.038 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:50.039 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:50.039 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2024-04-18T22:09:50.039 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-18T22:09:50.086 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.Jvz4oWHKqT 2024-04-18T22:09:50.086 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Jvz4oWHKqT 2024-04-18T22:09:50.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-04-18T22:09:50.131 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.E3LuXkWrKO is 10KB 2024-04-18T22:09:50.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.Jvz4oWHKqT 2024-04-18T22:09:50.144 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.E3LuXkWrKO 2024-04-18T22:09:50.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:50.202 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:09:50.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:09:50.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:50.263 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.Jvz4oWHKqT is 10KB 2024-04-18T22:09:50.264 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:09:50.264 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:09:50.270 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-18T22:09:50.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo update-grub 2024-04-18T22:09:50.276 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:09:50.276 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.Jvz4oWHKqT 2024-04-18T22:09:50.281 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:50.297 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:09:50.338 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:09:50.338 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:09:50.338 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:50.404 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-18T22:09:50.404 DEBUG:teuthology.orchestra.run.smithi196:> sudo update-grub 2024-04-18T22:09:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:09:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:51.432 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-18T22:09:51.510 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:09:51.510 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:09:51.525 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:51.544 INFO:teuthology.orchestra.run.smithi032.stdout:Generating grub configuration file ... 2024-04-18T22:09:51.615 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:09:51.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:09:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:51.685 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:09:51.791 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:09:51.792 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:51.794 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-04-18T22:09:51.862 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-04-18T22:09:51.863 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-04-18T22:09:51.880 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:51.881 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:51.887 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:51.902 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:51.952 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:51.958 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:52.014 INFO:teuthology.orchestra.run.smithi096.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:09:52.015 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:52.051 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:52.053 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-04-18T22:09:52.067 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:52.067 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:52.067 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-18T22:09:52.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-04-18T22:09:52.115 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:52.117 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-04-18T22:09:52.117 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-04-18T22:09:52.161 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.D1MvuUmpoM 2024-04-18T22:09:52.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub/grub.cfg /tmp/tmp.D1MvuUmpoM 2024-04-18T22:09:52.193 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:52.199 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:52.219 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.D1MvuUmpoM 2024-04-18T22:09:52.241 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:52.246 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:52.260 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:52.267 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:52.333 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.D1MvuUmpoM is 10KB 2024-04-18T22:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:52.345 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:52.346 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.D1MvuUmpoM 2024-04-18T22:09:52.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:52.353 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:52.354 INFO:teuthology.orchestra.run.smithi196.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:52.354 INFO:teuthology.orchestra.run.smithi196.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:52.384 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-04-18T22:09:52.393 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-04-18T22:09:52.407 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:09:52.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:09:52.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:52.473 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-18T22:09:52.473 DEBUG:teuthology.orchestra.run.smithi096:> sudo update-grub 2024-04-18T22:09:52.527 INFO:teuthology.orchestra.run.smithi032.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:52.527 INFO:teuthology.orchestra.run.smithi032.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:52.527 INFO:teuthology.orchestra.run.smithi032.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:52.574 INFO:teuthology.orchestra.run.smithi032.stdout:done 2024-04-18T22:09:53.078 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2024-04-18T22:09:53.143 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2024-04-18T22:09:53.143 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 21 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2024-04-18T22:09:53.145 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-18T22:09:53.145 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:09:53.145 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:09:53.145 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-18T22:09:53.145 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:09:53.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2024-04-18T22:09:53.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-04-18T22:09:53.463 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:09:53.475 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:09:53.476 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:09:53.496 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:09:53.793 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:09:53.795 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:53.798 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-04-18T22:09:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:54.064 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:54.336 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:54.341 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:54.386 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:54.391 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:54.523 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:54.535 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-04-18T22:09:54.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-04-18T22:09:54.912 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:09:54.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:09:54.995 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:09:55.198 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-04-18T22:09:55.198 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-04-18T22:09:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-18T22:09:55.381 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:09:55.381 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:09:55.382 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:09:55.382 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:09:55.382 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:09:55.382 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:09:55.382 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:09:55.422 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:09:55.425 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2024-04-18T22:09:55.441 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2024-04-18T22:09:55.441 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 21 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.15.0.102.99 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2024-04-18T22:09:55.442 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:09:55.443 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2024-04-18T22:09:55.443 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-18T22:09:55.487 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.dU0AnBAKO4 2024-04-18T22:09:55.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub/grub.cfg /tmp/tmp.dU0AnBAKO4 2024-04-18T22:09:55.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.dU0AnBAKO4 2024-04-18T22:09:55.668 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.dU0AnBAKO4 is 10KB 2024-04-18T22:09:55.681 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.dU0AnBAKO4 2024-04-18T22:09:55.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:55.744 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:09:55.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:09:55.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:09:55.830 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-18T22:09:55.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo update-grub 2024-04-18T22:09:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:09:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:09:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-04-18T22:09:57.443 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-18T22:09:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-18T22:09:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:09:57.729 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:09:57.771 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:09:57.777 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:09:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:09:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:09:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:09:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-04-18T22:09:57.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-04-18T22:10:22.110 INFO:teuthology.task.kernel:Checking client osd.0 for new kernel version... 2024-04-18T22:10:22.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-18T22:10:22.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:10:22.397 INFO:teuthology.task.kernel:Checking client osd.2 for new kernel version... 2024-04-18T22:10:22.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T22:10:22.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:10:24.541 INFO:teuthology.task.kernel:Checking client osd.1 for new kernel version... 2024-04-18T22:10:24.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:10:24.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:10:27.977 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-18T22:10:27.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:10:27.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:10:56.675 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-18T22:10:59.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-04-18T22:11:02.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-18T22:11:05.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-18T22:11:05.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:08.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:11:08.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:11.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:11:11.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:14.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-18T22:11:22.406 DEBUG:teuthology.orchestra.remote:timed out 2024-04-18T22:11:24.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-18T22:11:26.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-04-18T22:11:26.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:11:26.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:29.695 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-18T22:11:31.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T22:11:31.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:31.936 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-18T22:11:33.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T22:11:33.122 INFO:teuthology.task.kernel:Checking kernel version of osd.2, want "5.15.0-102-generic"... 2024-04-18T22:11:33.122 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-18T22:11:33.128 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-102-generic 2024-04-18T22:11:33.128 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-18T22:11:33.128 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:11:33.128 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:11:34.129 INFO:teuthology.task.kernel:Enabling kdb on osd.2... 2024-04-18T22:11:34.129 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:11:34.544 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-04-18T22:11:34.559 DEBUG:teuthology.parallel:result is None 2024-04-18T22:11:36.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-18T22:11:36.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:36.381 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-18T22:11:37.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-18T22:11:37.309 INFO:teuthology.task.kernel:Checking kernel version of osd.0, want "5.15.0-102-generic"... 2024-04-18T22:11:37.309 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-18T22:11:37.361 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-102-generic 2024-04-18T22:11:37.361 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-18T22:11:37.361 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:11:37.361 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:11:38.362 INFO:teuthology.task.kernel:Enabling kdb on osd.0... 2024-04-18T22:11:38.362 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:11:38.439 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-04-18T22:11:38.467 DEBUG:teuthology.parallel:result is None 2024-04-18T22:11:38.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:11:38.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:38.953 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-18T22:11:40.142 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:11:40.142 INFO:teuthology.task.kernel:Checking kernel version of osd.1, want "5.15.0-102-generic"... 2024-04-18T22:11:40.142 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-18T22:11:40.202 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-102-generic 2024-04-18T22:11:40.202 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-18T22:11:40.203 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:11:40.203 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:11:41.204 INFO:teuthology.task.kernel:Enabling kdb on osd.1... 2024-04-18T22:11:41.204 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:11:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-04-18T22:11:41.334 DEBUG:teuthology.parallel:result is None 2024-04-18T22:11:44.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:11:44.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:11:44.963 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-18T22:11:45.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:11:45.624 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-102-generic"... 2024-04-18T22:11:45.625 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-18T22:11:45.672 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-102-generic 2024-04-18T22:11:45.672 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-18T22:11:45.672 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:11:45.672 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:11:46.673 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-18T22:11:46.673 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:11:46.755 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2024-04-18T22:11:46.778 DEBUG:teuthology.parallel:result is None 2024-04-18T22:11:46.778 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-18T22:11:46.785 INFO:teuthology.task.internal:Creating test directory... 2024-04-18T22:11:46.785 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:11:46.787 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:11:46.790 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:11:46.793 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:11:46.799 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-18T22:11:46.999 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-18T22:11:47.037 INFO:teuthology.task.internal:Creating archive directory... 2024-04-18T22:11:47.037 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:11:47.040 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:11:47.042 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:11:47.044 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:11:47.072 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-18T22:11:47.094 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-18T22:11:47.094 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:11:47.097 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:11:47.106 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:11:47.108 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:11:47.117 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.122 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.125 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.127 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.129 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.130 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.135 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.137 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:11:47.138 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-18T22:11:47.145 INFO:teuthology.task.internal:Configuring sudo... 2024-04-18T22:11:47.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:11:47.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:11:47.174 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:11:47.181 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:11:47.195 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-18T22:11:47.240 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-18T22:11:47.240 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:11:47.243 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:11:47.245 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:11:47.249 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:11:47.254 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:11:47.293 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:11:47.342 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:11:47.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:11:47.404 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:11:47.409 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:11:47.458 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:11:47.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:11:47.518 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:11:47.524 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:11:47.572 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:11:47.572 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:11:47.638 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:11:47.644 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:11:47.694 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:11:47.694 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:11:47.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-04-18T22:11:47.769 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-18T22:11:47.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-04-18T22:11:47.773 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-18T22:11:47.844 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-18T22:11:47.852 INFO:teuthology.task.internal:Starting timer... 2024-04-18T22:11:47.852 INFO:teuthology.run_tasks:Running task pcp... 2024-04-18T22:11:47.957 INFO:teuthology.run_tasks:Running task selinux... 2024-04-18T22:11:48.019 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2024-04-18T22:11:48.019 DEBUG:teuthology.task.selinux:Excluding smithi086: OS 'ubuntu' does not support SELinux 2024-04-18T22:11:48.019 DEBUG:teuthology.task.selinux:Excluding smithi096: OS 'ubuntu' does not support SELinux 2024-04-18T22:11:48.019 DEBUG:teuthology.task.selinux:Excluding smithi196: OS 'ubuntu' does not support SELinux 2024-04-18T22:11:48.019 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-18T22:11:48.019 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-18T22:11:48.019 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-18T22:11:48.019 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-18T22:11:48.056 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-18T22:11:48.071 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-18T22:11:48.072 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi086.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-18T22:18:59.109 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-04-18T22:18:59.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:18:59.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:18:59.191 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-18T22:18:59.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-18T22:18:59.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-18T22:18:59.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:18:59.354 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-18T22:18:59.430 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-18T22:18:59.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:18:59.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:18:59.510 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-18T22:18:59.586 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:18:59.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T22:18:59.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:18:59.665 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-18T22:18:59.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T22:18:59.740 INFO:teuthology.run_tasks:Running task clock... 2024-04-18T22:18:59.751 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-18T22:18:59.751 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T22:18:59.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:18:59.754 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T22:18:59.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:18:59.757 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T22:18:59.757 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:18:59.759 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T22:18:59.759 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:18:59.781 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T22:18:59.781 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Command line: ntpd -gq 2024-04-18T22:18:59.782 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: ---------------------------------------------------- 2024-04-18T22:18:59.782 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T22:18:59.782 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T22:18:59.782 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: corporation. Support and training for ntp-4 are 2024-04-18T22:18:59.782 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: available at https://www.nwtime.org/support 2024-04-18T22:18:59.782 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: ---------------------------------------------------- 2024-04-18T22:18:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: proto: precision = 0.057 usec (-24) 2024-04-18T22:18:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: basedate set to 2022-02-04 2024-04-18T22:18:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: gps base set to 2022-02-06 (week 2196) 2024-04-18T22:18:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.783 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.784 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T22:18:59.785 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T22:18:59.785 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T22:18:59.785 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listen normally on 3 enp3s0f1 172.21.15.86:123 2024-04-18T22:18:59.785 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listen normally on 4 lo [::1]:123 2024-04-18T22:18:59.785 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d809%5]:123 2024-04-18T22:18:59.785 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:18:59 ntpd[17241]: Listening on routing socket on fd #22 for interface updates 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Command line: ntpd -gq 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: ---------------------------------------------------- 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: corporation. Support and training for ntp-4 are 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: available at https://www.nwtime.org/support 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: ---------------------------------------------------- 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: proto: precision = 0.064 usec (-24) 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: basedate set to 2022-02-04 2024-04-18T22:18:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: gps base set to 2022-02-06 (week 2196) 2024-04-18T22:18:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.787 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.787 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.788 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T22:18:59.788 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T22:18:59.788 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T22:18:59.788 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listen normally on 3 ens1f0 172.21.15.32:123 2024-04-18T22:18:59.788 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listen normally on 4 lo [::1]:123 2024-04-18T22:18:59.788 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%4]:123 2024-04-18T22:18:59.789 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:18:59 ntpd[17083]: Listening on routing socket on fd #22 for interface updates 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Command line: ntpd -gq 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: ---------------------------------------------------- 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: corporation. Support and training for ntp-4 are 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: available at https://www.nwtime.org/support 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: ---------------------------------------------------- 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: proto: precision = 0.044 usec (-24) 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: basedate set to 2022-02-04 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: gps base set to 2022-02-06 (week 2196) 2024-04-18T22:18:59.790 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.791 INFO:teuthology.orchestra.run.smithi096.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.791 INFO:teuthology.orchestra.run.smithi096.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listen normally on 3 enp3s0f1 172.21.15.96:123 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listen normally on 4 lo [::1]:123 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc0d%5]:123 2024-04-18T22:18:59.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:18:59 ntpd[17226]: Listening on routing socket on fd #22 for interface updates 2024-04-18T22:18:59.816 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T22:18:59.816 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Command line: ntpd -gq 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: ---------------------------------------------------- 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: corporation. Support and training for ntp-4 are 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: available at https://www.nwtime.org/support 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: ---------------------------------------------------- 2024-04-18T22:18:59.817 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: proto: precision = 0.091 usec (-23) 2024-04-18T22:18:59.818 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: basedate set to 2022-02-04 2024-04-18T22:18:59.818 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: gps base set to 2022-02-06 (week 2196) 2024-04-18T22:18:59.818 INFO:teuthology.orchestra.run.smithi196.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.818 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T22:18:59.819 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.819 INFO:teuthology.orchestra.run.smithi196.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listen normally on 3 enp3s0f1 172.21.15.196:123 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listen normally on 4 lo [::1]:123 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f47%5]:123 2024-04-18T22:18:59.820 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:18:59 ntpd[17182]: Listening on routing socket on fd #22 for interface updates 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:00.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:00 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:01 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:01 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:01 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:01 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:01 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:01 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:01 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:01.789 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:01 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.789 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:01 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.789 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:01 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:01.789 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:01 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:01.789 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:01 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:01.789 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:01 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:02 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:02 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:02 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:02 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:02 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:02 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:02.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:02.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:02 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:03 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:03.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:03.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:03 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:03 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:04 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:04 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:04 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:04 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:04 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:04.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:04 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:04 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:04.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:04.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:04 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:05 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:05 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:05 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:05 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:05 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:05 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:05 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:05 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:06 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:06 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:06 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:06 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:06 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:06.782 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:06 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:06 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:06.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.817 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:06.818 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:06.819 INFO:teuthology.orchestra.run.smithi196.stdout:18 Apr 22:19:06 ntpd[17182]: ntpd: time slew -0.000803 s 2024-04-18T22:19:06.819 INFO:teuthology.orchestra.run.smithi196.stdout:ntpd: time slew -0.000803s 2024-04-18T22:19:06.819 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T22:19:06.819 INFO:teuthology.orchestra.run.smithi196.stderr:18 Apr 22:19:06 ntpd[17182]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T22:19:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:19:06.944 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-04-18T22:19:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:07.784 INFO:teuthology.orchestra.run.smithi086.stdout:18 Apr 22:19:07 ntpd[17241]: ntpd: time slew -0.001078 s 2024-04-18T22:19:07.784 INFO:teuthology.orchestra.run.smithi086.stdout:ntpd: time slew -0.001078s 2024-04-18T22:19:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T22:19:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:18 Apr 22:19:07 ntpd[17241]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T22:19:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stdout:18 Apr 22:19:07 ntpd[17083]: ntpd: time slew +0.000148 s 2024-04-18T22:19:07.786 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew +0.000148s 2024-04-18T22:19:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T22:19:07.787 INFO:teuthology.orchestra.run.smithi032.stderr:18 Apr 22:19:07 ntpd[17083]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T22:19:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T22:19:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T22:19:07.792 INFO:teuthology.orchestra.run.smithi096.stdout:18 Apr 22:19:07 ntpd[17226]: ntpd: time slew -0.002489 s 2024-04-18T22:19:07.792 INFO:teuthology.orchestra.run.smithi096.stdout:ntpd: time slew -0.002489s 2024-04-18T22:19:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T22:19:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:18 Apr 22:19:07 ntpd[17226]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T22:19:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:19:07.833 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-04-18T22:19:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:19:07.836 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-04-18T22:19:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.846 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:19:07.847 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-04-18T22:19:07.847 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.847 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.847 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.847 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:19:07.848 INFO:teuthology.run_tasks:Running task install... 2024-04-18T22:19:07.857 DEBUG:teuthology.task.install:project ceph 2024-04-18T22:19:07.857 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}} 2024-04-18T22:19:07.857 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'} 2024-04-18T22:19:07.857 INFO:teuthology.task.install:Using flavor: default 2024-04-18T22:19:07.862 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-18T22:19:07.862 INFO:teuthology.task.install:extra packages: [] 2024-04-18T22:19:07.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2024-04-18T22:19:07.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-key list | grep Ceph 2024-04-18T22:19:07.863 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-key list | grep Ceph 2024-04-18T22:19:07.864 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-key list | grep Ceph 2024-04-18T22:19:07.927 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T22:19:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T22:19:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T22:19:07.943 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T22:19:07.956 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T22:19:07.956 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T22:19:07.957 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T22:19:07.957 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-04-18T22:19:07.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:19:07.962 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T22:19:07.962 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T22:19:07.962 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T22:19:07.962 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-04-18T22:19:07.963 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:19:07.970 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T22:19:07.970 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T22:19:07.970 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T22:19:07.970 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-04-18T22:19:07.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:19:07.974 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T22:19:07.974 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T22:19:07.974 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T22:19:07.974 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-04-18T22:19:07.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:19:08.096 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T22:19:08.097 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:08.101 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T22:19:08.101 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:08.202 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:19:08.202 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T22:19:08.209 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:19:08.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T22:19:08.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-18T22:19:08.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2024-04-18T22:19:08.277 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T22:19:08.278 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:08.294 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T22:19:08.294 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:08.390 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:19:08.390 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:19:08.396 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:19:08.396 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:19:08.404 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:19:08.404 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T22:19:08.405 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:19:08.409 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:19:08.418 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-04-18T22:19:08.429 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:19:08.433 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T22:19:08.435 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:19:08.449 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:19:08.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T22:19:08.454 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T22:19:08.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-04-18T22:19:08.476 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T22:19:08.497 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T22:19:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T22:19:08.524 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T22:19:08.587 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T22:19:08.643 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:19:08.643 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:19:08.643 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T22:19:08.644 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T22:19:08.664 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T22:19:08.686 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T22:19:08.686 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:19:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:19:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:19:08.706 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T22:19:08.722 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T22:19:08.733 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 29.0 kB in 0s (84.6 kB/s) 2024-04-18T22:19:08.734 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T22:19:08.744 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:19:08.745 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:19:08.801 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:19:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T22:19:08.862 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T22:19:08.893 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 29.0 kB in 0s (61.1 kB/s) 2024-04-18T22:19:08.923 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T22:19:08.990 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T22:19:09.020 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 29.0 kB in 0s (68.7 kB/s) 2024-04-18T22:19:09.057 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T22:19:09.133 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T22:19:09.223 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 29.0 kB in 1s (51.0 kB/s) 2024-04-18T22:19:09.723 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:19:09.739 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:09.799 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:19:09.860 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:19:09.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:09.934 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:19:09.994 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-18T22:19:09.995 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-18T22:19:10.017 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:19:10.032 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:10.090 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:19:10.133 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-18T22:19:10.134 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-18T22:19:10.180 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:19:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:19:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:19:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:19:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:19:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:19:10.181 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T22:19:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet zip 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-04-18T22:19:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: btrfs-tools 2024-04-18T22:19:10.220 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:19:10.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:19:10.283 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-18T22:19:10.283 INFO:teuthology.orchestra.run.smithi196.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T22:19:10.283 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T22:19:10.283 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T22:19:10.283 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T22:19:10.283 INFO:teuthology.orchestra.run.smithi196.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T22:19:10.284 INFO:teuthology.orchestra.run.smithi196.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.285 INFO:teuthology.orchestra.run.smithi196.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T22:19:10.286 INFO:teuthology.orchestra.run.smithi196.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T22:19:10.287 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-04-18T22:19:10.287 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-04-18T22:19:10.288 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:19:10.315 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:19:10.316 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:19:10.316 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:19:10.316 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:19:10.316 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:19:10.316 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:19:10.317 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:19:10.317 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-18T22:19:10.317 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T22:19:10.318 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T22:19:10.318 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T22:19:10.318 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T22:19:10.318 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T22:19:10.318 INFO:teuthology.orchestra.run.smithi086.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T22:19:10.318 INFO:teuthology.orchestra.run.smithi086.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T22:19:10.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet zip 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T22:19:10.320 INFO:teuthology.orchestra.run.smithi086.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T22:19:10.321 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2024-04-18T22:19:10.321 INFO:teuthology.orchestra.run.smithi086.stdout: btrfs-tools 2024-04-18T22:19:10.351 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T22:19:10.351 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 179 MB of archives. 2024-04-18T22:19:10.352 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T22:19:10.352 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T22:19:10.419 INFO:teuthology.orchestra.run.smithi086.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T22:19:10.422 INFO:teuthology.orchestra.run.smithi086.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T22:19:10.468 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-04-18T22:19:10.468 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-04-18T22:19:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T22:19:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T22:19:10.479 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:19:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:19:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:19:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:19:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:19:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:19:10.480 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:19:10.481 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-04-18T22:19:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T22:19:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T22:19:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T22:19:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T22:19:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet zip 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T22:19:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T22:19:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T22:19:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T22:19:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T22:19:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T22:19:10.485 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-04-18T22:19:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: btrfs-tools 2024-04-18T22:19:10.494 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T22:19:10.495 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 179 MB of archives. 2024-04-18T22:19:10.495 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T22:19:10.495 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T22:19:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T22:19:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T22:19:10.584 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-04-18T22:19:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T22:19:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T22:19:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T22:19:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T22:19:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T22:19:10.585 INFO:teuthology.orchestra.run.smithi096.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T22:19:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T22:19:10.587 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T22:19:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T22:19:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T22:19:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T22:19:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T22:19:10.588 INFO:teuthology.orchestra.run.smithi096.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T22:19:10.588 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T22:19:10.594 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T22:19:10.611 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T22:19:10.616 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T22:19:10.617 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T22:19:10.618 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T22:19:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T22:19:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T22:19:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:19:10.653 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:19:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T22:19:10.655 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-04-18T22:19:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T22:19:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet zip 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T22:19:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-04-18T22:19:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2024-04-18T22:19:10.670 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T22:19:10.680 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T22:19:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T22:19:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T22:19:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T22:19:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T22:19:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T22:19:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T22:19:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T22:19:10.716 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T22:19:10.716 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 179 MB of archives. 2024-04-18T22:19:10.716 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T22:19:10.716 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T22:19:10.735 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T22:19:10.742 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T22:19:10.742 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T22:19:10.743 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T22:19:10.743 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T22:19:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-04-18T22:19:10.759 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T22:19:10.759 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T22:19:10.759 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T22:19:10.760 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T22:19:10.760 INFO:teuthology.orchestra.run.smithi032.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T22:19:10.760 INFO:teuthology.orchestra.run.smithi032.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T22:19:10.760 INFO:teuthology.orchestra.run.smithi032.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T22:19:10.761 INFO:teuthology.orchestra.run.smithi032.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T22:19:10.762 INFO:teuthology.orchestra.run.smithi032.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T22:19:10.763 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T22:19:10.763 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T22:19:10.763 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T22:19:10.768 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T22:19:10.769 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T22:19:10.770 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T22:19:10.771 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T22:19:10.782 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T22:19:10.782 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T22:19:10.783 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T22:19:10.808 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T22:19:10.810 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T22:19:10.811 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T22:19:10.822 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T22:19:10.822 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T22:19:10.824 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T22:19:10.825 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T22:19:10.843 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T22:19:10.850 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T22:19:10.850 INFO:teuthology.orchestra.run.smithi196.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T22:19:10.851 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T22:19:10.851 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 179 MB of archives. 2024-04-18T22:19:10.851 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T22:19:10.851 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T22:19:10.867 INFO:teuthology.orchestra.run.smithi196.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T22:19:10.869 INFO:teuthology.orchestra.run.smithi196.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T22:19:10.873 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T22:19:10.874 INFO:teuthology.orchestra.run.smithi196.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T22:19:10.875 INFO:teuthology.orchestra.run.smithi196.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T22:19:10.876 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T22:19:10.877 INFO:teuthology.orchestra.run.smithi196.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T22:19:10.882 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T22:19:10.882 INFO:teuthology.orchestra.run.smithi196.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T22:19:10.909 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T22:19:10.909 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T22:19:10.915 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T22:19:10.916 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T22:19:10.918 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T22:19:10.919 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T22:19:10.936 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T22:19:10.942 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T22:19:10.950 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T22:19:10.953 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T22:19:10.983 INFO:teuthology.orchestra.run.smithi196.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T22:19:10.984 INFO:teuthology.orchestra.run.smithi196.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T22:19:10.985 INFO:teuthology.orchestra.run.smithi196.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T22:19:10.986 INFO:teuthology.orchestra.run.smithi196.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T22:19:10.988 INFO:teuthology.orchestra.run.smithi196.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T22:19:10.994 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T22:19:10.996 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T22:19:11.000 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T22:19:11.004 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T22:19:11.008 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T22:19:11.009 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T22:19:11.036 INFO:teuthology.orchestra.run.smithi196.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T22:19:11.039 INFO:teuthology.orchestra.run.smithi196.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T22:19:11.040 INFO:teuthology.orchestra.run.smithi196.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T22:19:11.058 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T22:19:11.060 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T22:19:11.060 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T22:19:11.061 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T22:19:11.062 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T22:19:11.086 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T22:19:11.087 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T22:19:11.087 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T22:19:11.093 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T22:19:11.094 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T22:19:11.096 INFO:teuthology.orchestra.run.smithi196.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T22:19:11.096 INFO:teuthology.orchestra.run.smithi196.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T22:19:11.097 INFO:teuthology.orchestra.run.smithi196.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T22:19:11.098 INFO:teuthology.orchestra.run.smithi196.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T22:19:11.098 INFO:teuthology.orchestra.run.smithi196.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T22:19:11.099 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T22:19:11.099 INFO:teuthology.orchestra.run.smithi196.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T22:19:11.099 INFO:teuthology.orchestra.run.smithi196.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T22:19:11.100 INFO:teuthology.orchestra.run.smithi196.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T22:19:11.101 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T22:19:11.104 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T22:19:11.105 INFO:teuthology.orchestra.run.smithi196.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T22:19:11.115 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T22:19:11.116 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T22:19:11.118 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T22:19:11.118 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T22:19:11.119 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T22:19:11.125 INFO:teuthology.orchestra.run.smithi196.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T22:19:11.127 INFO:teuthology.orchestra.run.smithi196.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T22:19:11.136 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T22:19:11.138 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T22:19:11.139 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T22:19:11.146 INFO:teuthology.orchestra.run.smithi196.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T22:19:11.150 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T22:19:11.151 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T22:19:11.152 INFO:teuthology.orchestra.run.smithi196.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T22:19:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T22:19:11.158 INFO:teuthology.orchestra.run.smithi196.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T22:19:11.159 INFO:teuthology.orchestra.run.smithi196.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T22:19:11.160 INFO:teuthology.orchestra.run.smithi196.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T22:19:11.169 INFO:teuthology.orchestra.run.smithi196.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T22:19:11.170 INFO:teuthology.orchestra.run.smithi196.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T22:19:11.172 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T22:19:11.173 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T22:19:11.175 INFO:teuthology.orchestra.run.smithi196.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T22:19:11.176 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T22:19:11.180 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T22:19:11.181 INFO:teuthology.orchestra.run.smithi086.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T22:19:11.187 INFO:teuthology.orchestra.run.smithi196.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T22:19:11.188 INFO:teuthology.orchestra.run.smithi196.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T22:19:11.195 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T22:19:11.207 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T22:19:11.220 INFO:teuthology.orchestra.run.smithi086.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T22:19:11.222 INFO:teuthology.orchestra.run.smithi086.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T22:19:11.227 INFO:teuthology.orchestra.run.smithi086.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T22:19:11.229 INFO:teuthology.orchestra.run.smithi086.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T22:19:11.230 INFO:teuthology.orchestra.run.smithi086.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T22:19:11.238 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T22:19:11.239 INFO:teuthology.orchestra.run.smithi196.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T22:19:11.242 INFO:teuthology.orchestra.run.smithi196.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T22:19:11.243 INFO:teuthology.orchestra.run.smithi196.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T22:19:11.257 INFO:teuthology.orchestra.run.smithi196.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T22:19:11.258 INFO:teuthology.orchestra.run.smithi196.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T22:19:11.258 INFO:teuthology.orchestra.run.smithi196.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T22:19:11.259 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T22:19:11.260 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T22:19:11.261 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T22:19:11.263 INFO:teuthology.orchestra.run.smithi196.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T22:19:11.264 INFO:teuthology.orchestra.run.smithi196.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T22:19:11.285 INFO:teuthology.orchestra.run.smithi086.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T22:19:11.342 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T22:19:11.346 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T22:19:11.350 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T22:19:11.353 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T22:19:11.353 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T22:19:11.358 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T22:19:11.362 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T22:19:11.366 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T22:19:11.367 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T22:19:11.395 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T22:19:11.405 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T22:19:11.406 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T22:19:11.406 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T22:19:11.408 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T22:19:11.408 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T22:19:11.409 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T22:19:11.409 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T22:19:11.415 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T22:19:11.428 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T22:19:11.439 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T22:19:11.441 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T22:19:11.442 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T22:19:11.444 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T22:19:11.444 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T22:19:11.445 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T22:19:11.447 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T22:19:11.449 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T22:19:11.450 INFO:teuthology.orchestra.run.smithi086.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T22:19:11.450 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T22:19:11.450 INFO:teuthology.orchestra.run.smithi086.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T22:19:11.452 INFO:teuthology.orchestra.run.smithi086.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T22:19:11.452 INFO:teuthology.orchestra.run.smithi086.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T22:19:11.454 INFO:teuthology.orchestra.run.smithi086.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T22:19:11.459 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T22:19:11.460 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T22:19:11.474 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T22:19:11.475 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T22:19:11.476 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T22:19:11.479 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T22:19:11.483 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T22:19:11.485 INFO:teuthology.orchestra.run.smithi096.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T22:19:11.492 INFO:teuthology.orchestra.run.smithi086.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T22:19:11.497 INFO:teuthology.orchestra.run.smithi086.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T22:19:11.497 INFO:teuthology.orchestra.run.smithi086.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T22:19:11.501 INFO:teuthology.orchestra.run.smithi096.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T22:19:11.503 INFO:teuthology.orchestra.run.smithi096.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T22:19:11.508 INFO:teuthology.orchestra.run.smithi096.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T22:19:11.509 INFO:teuthology.orchestra.run.smithi096.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T22:19:11.544 INFO:teuthology.orchestra.run.smithi086.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T22:19:11.544 INFO:teuthology.orchestra.run.smithi086.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T22:19:11.545 INFO:teuthology.orchestra.run.smithi086.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T22:19:11.546 INFO:teuthology.orchestra.run.smithi086.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T22:19:11.546 INFO:teuthology.orchestra.run.smithi096.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T22:19:11.547 INFO:teuthology.orchestra.run.smithi086.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T22:19:11.548 INFO:teuthology.orchestra.run.smithi086.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T22:19:11.551 INFO:teuthology.orchestra.run.smithi096.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T22:19:11.558 INFO:teuthology.orchestra.run.smithi086.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T22:19:11.559 INFO:teuthology.orchestra.run.smithi086.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T22:19:11.575 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T22:19:11.575 INFO:teuthology.orchestra.run.smithi086.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T22:19:11.600 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T22:19:11.601 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T22:19:11.606 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T22:19:11.608 INFO:teuthology.orchestra.run.smithi086.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T22:19:11.612 INFO:teuthology.orchestra.run.smithi086.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T22:19:11.614 INFO:teuthology.orchestra.run.smithi086.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T22:19:11.623 INFO:teuthology.orchestra.run.smithi086.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T22:19:11.630 INFO:teuthology.orchestra.run.smithi086.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T22:19:11.632 INFO:teuthology.orchestra.run.smithi086.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T22:19:11.634 INFO:teuthology.orchestra.run.smithi086.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T22:19:11.661 INFO:teuthology.orchestra.run.smithi086.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T22:19:11.663 INFO:teuthology.orchestra.run.smithi086.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T22:19:11.671 INFO:teuthology.orchestra.run.smithi086.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T22:19:11.678 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T22:19:11.683 INFO:teuthology.orchestra.run.smithi096.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T22:19:11.685 INFO:teuthology.orchestra.run.smithi096.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T22:19:11.687 INFO:teuthology.orchestra.run.smithi096.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T22:19:11.687 INFO:teuthology.orchestra.run.smithi096.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T22:19:11.689 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T22:19:11.689 INFO:teuthology.orchestra.run.smithi096.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T22:19:11.705 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T22:19:11.706 INFO:teuthology.orchestra.run.smithi086.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T22:19:11.708 INFO:teuthology.orchestra.run.smithi086.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T22:19:11.709 INFO:teuthology.orchestra.run.smithi086.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T22:19:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T22:19:11.715 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T22:19:11.716 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T22:19:11.716 INFO:teuthology.orchestra.run.smithi086.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T22:19:11.727 INFO:teuthology.orchestra.run.smithi096.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T22:19:11.732 INFO:teuthology.orchestra.run.smithi096.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T22:19:11.732 INFO:teuthology.orchestra.run.smithi096.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T22:19:11.734 INFO:teuthology.orchestra.run.smithi086.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T22:19:11.736 INFO:teuthology.orchestra.run.smithi086.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T22:19:11.752 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T22:19:11.763 INFO:teuthology.orchestra.run.smithi096.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T22:19:11.763 INFO:teuthology.orchestra.run.smithi096.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T22:19:11.765 INFO:teuthology.orchestra.run.smithi086.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T22:19:11.770 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T22:19:11.771 INFO:teuthology.orchestra.run.smithi086.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T22:19:11.774 INFO:teuthology.orchestra.run.smithi086.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T22:19:11.783 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T22:19:11.787 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T22:19:11.788 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T22:19:11.793 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T22:19:11.794 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T22:19:11.794 INFO:teuthology.orchestra.run.smithi096.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T22:19:11.794 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T22:19:11.795 INFO:teuthology.orchestra.run.smithi096.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T22:19:11.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T22:19:11.796 INFO:teuthology.orchestra.run.smithi096.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T22:19:11.796 INFO:teuthology.orchestra.run.smithi096.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T22:19:11.797 INFO:teuthology.orchestra.run.smithi096.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T22:19:11.798 INFO:teuthology.orchestra.run.smithi096.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T22:19:11.803 INFO:teuthology.orchestra.run.smithi096.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T22:19:11.813 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T22:19:11.814 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T22:19:11.814 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T22:19:11.815 INFO:teuthology.orchestra.run.smithi096.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T22:19:11.817 INFO:teuthology.orchestra.run.smithi096.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T22:19:11.818 INFO:teuthology.orchestra.run.smithi096.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T22:19:11.836 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T22:19:11.836 INFO:teuthology.orchestra.run.smithi096.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T22:19:11.841 INFO:teuthology.orchestra.run.smithi096.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T22:19:11.842 INFO:teuthology.orchestra.run.smithi096.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T22:19:11.842 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T22:19:11.843 INFO:teuthology.orchestra.run.smithi096.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T22:19:11.846 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T22:19:11.848 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T22:19:11.849 INFO:teuthology.orchestra.run.smithi096.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T22:19:11.850 INFO:teuthology.orchestra.run.smithi096.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T22:19:11.854 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T22:19:11.854 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T22:19:11.855 INFO:teuthology.orchestra.run.smithi096.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T22:19:11.855 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T22:19:11.860 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T22:19:11.862 INFO:teuthology.orchestra.run.smithi096.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T22:19:11.863 INFO:teuthology.orchestra.run.smithi096.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T22:19:11.864 INFO:teuthology.orchestra.run.smithi096.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T22:19:11.866 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T22:19:11.880 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T22:19:11.884 INFO:teuthology.orchestra.run.smithi096.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T22:19:11.885 INFO:teuthology.orchestra.run.smithi096.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T22:19:11.897 INFO:teuthology.orchestra.run.smithi096.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T22:19:11.897 INFO:teuthology.orchestra.run.smithi096.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T22:19:11.898 INFO:teuthology.orchestra.run.smithi096.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T22:19:11.901 INFO:teuthology.orchestra.run.smithi096.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T22:19:11.902 INFO:teuthology.orchestra.run.smithi096.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T22:19:11.932 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T22:19:11.932 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T22:19:11.934 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T22:19:11.935 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T22:19:11.939 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T22:19:11.954 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T22:19:11.959 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T22:19:11.969 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T22:19:11.974 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T22:19:11.993 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T22:19:11.997 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T22:19:12.003 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T22:19:12.014 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T22:19:12.112 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T22:19:12.163 INFO:teuthology.orchestra.run.smithi096.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T22:19:12.367 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T22:19:12.370 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T22:19:12.376 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T22:19:12.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T22:19:12.382 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T22:19:12.407 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T22:19:12.482 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T22:19:12.486 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T22:19:12.494 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T22:19:12.494 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T22:19:12.495 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T22:19:12.497 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T22:19:12.514 INFO:teuthology.orchestra.run.smithi096.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T22:19:12.525 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T22:19:12.527 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T22:19:12.539 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T22:19:12.554 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T22:19:12.554 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T22:19:12.556 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T22:19:12.557 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T22:19:12.558 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T22:19:12.560 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T22:19:12.561 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T22:19:12.562 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T22:19:12.569 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T22:19:12.576 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T22:19:12.592 INFO:teuthology.orchestra.run.smithi096.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T22:19:12.593 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T22:19:12.594 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T22:19:12.608 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T22:19:12.613 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T22:19:12.616 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T22:19:12.617 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T22:19:12.628 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T22:19:12.629 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T22:19:12.633 INFO:teuthology.orchestra.run.smithi096.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T22:19:12.634 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T22:19:12.634 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T22:19:12.635 INFO:teuthology.orchestra.run.smithi096.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T22:19:12.636 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T22:19:12.640 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T22:19:12.640 INFO:teuthology.orchestra.run.smithi096.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T22:19:12.642 INFO:teuthology.orchestra.run.smithi096.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T22:19:12.643 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T22:19:12.645 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T22:19:12.665 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T22:19:12.666 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T22:19:12.684 INFO:teuthology.orchestra.run.smithi096.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T22:19:12.795 INFO:teuthology.orchestra.run.smithi196.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T22:19:13.043 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T22:19:13.073 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T22:19:13.092 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T22:19:13.318 INFO:teuthology.orchestra.run.smithi096.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T22:19:13.321 INFO:teuthology.orchestra.run.smithi096.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T22:19:13.335 INFO:teuthology.orchestra.run.smithi096.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T22:19:13.465 INFO:teuthology.orchestra.run.smithi086.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T22:19:14.951 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T22:19:15.203 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T22:19:15.211 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T22:19:15.217 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T22:19:15.337 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T22:19:15.580 INFO:teuthology.orchestra.run.smithi096.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T22:19:15.705 INFO:teuthology.orchestra.run.smithi196.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T22:19:15.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T22:19:15.990 INFO:teuthology.orchestra.run.smithi096.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T22:19:16.012 INFO:teuthology.orchestra.run.smithi096.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T22:19:16.015 INFO:teuthology.orchestra.run.smithi096.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T22:19:16.090 INFO:teuthology.orchestra.run.smithi096.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T22:19:16.154 INFO:teuthology.orchestra.run.smithi086.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T22:19:16.341 INFO:teuthology.orchestra.run.smithi196.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T22:19:16.624 INFO:teuthology.orchestra.run.smithi096.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T22:19:16.644 INFO:teuthology.orchestra.run.smithi196.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T22:19:16.730 INFO:teuthology.orchestra.run.smithi196.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T22:19:16.777 INFO:teuthology.orchestra.run.smithi086.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T22:19:16.881 INFO:teuthology.orchestra.run.smithi196.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T22:19:16.917 INFO:teuthology.orchestra.run.smithi196.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T22:19:17.056 INFO:teuthology.orchestra.run.smithi086.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T22:19:17.057 INFO:teuthology.orchestra.run.smithi086.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T22:19:17.244 INFO:teuthology.orchestra.run.smithi086.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T22:19:17.246 INFO:teuthology.orchestra.run.smithi086.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T22:19:17.290 INFO:teuthology.orchestra.run.smithi196.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T22:19:17.410 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T22:19:17.411 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T22:19:17.433 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T22:19:17.505 INFO:teuthology.orchestra.run.smithi086.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T22:19:17.572 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T22:19:17.652 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T22:19:17.656 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T22:19:17.830 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T22:19:18.644 INFO:teuthology.orchestra.run.smithi032.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T22:19:18.645 INFO:teuthology.orchestra.run.smithi032.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T22:19:19.034 INFO:teuthology.orchestra.run.smithi096.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T22:19:19.034 INFO:teuthology.orchestra.run.smithi096.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T22:19:19.070 INFO:teuthology.orchestra.run.smithi096.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T22:19:19.384 INFO:teuthology.orchestra.run.smithi096.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T22:19:19.470 INFO:teuthology.orchestra.run.smithi096.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T22:19:19.508 INFO:teuthology.orchestra.run.smithi096.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T22:19:19.803 INFO:teuthology.orchestra.run.smithi096.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T22:19:21.038 INFO:teuthology.orchestra.run.smithi096.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T22:19:21.039 INFO:teuthology.orchestra.run.smithi096.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T22:19:22.894 INFO:teuthology.orchestra.run.smithi086.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T22:19:23.003 INFO:teuthology.orchestra.run.smithi086.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T22:19:23.403 INFO:teuthology.orchestra.run.smithi086.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T22:19:23.797 INFO:teuthology.orchestra.run.smithi196.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T22:19:23.882 INFO:teuthology.orchestra.run.smithi196.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T22:19:24.308 INFO:teuthology.orchestra.run.smithi196.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T22:19:25.006 INFO:teuthology.orchestra.run.smithi032.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T22:19:25.008 INFO:teuthology.orchestra.run.smithi032.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T22:19:25.009 INFO:teuthology.orchestra.run.smithi032.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T22:19:26.552 INFO:teuthology.orchestra.run.smithi032.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T22:19:27.135 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 179 MB in 16s (11.4 MB/s) 2024-04-18T22:19:27.303 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T22:19:27.400 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T22:19:27.406 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T22:19:27.494 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:19:28.056 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T22:19:28.082 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T22:19:28.122 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:19:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T22:19:28.493 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T22:19:28.533 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:19:28.674 INFO:teuthology.orchestra.run.smithi096.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T22:19:28.676 INFO:teuthology.orchestra.run.smithi096.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T22:19:28.677 INFO:teuthology.orchestra.run.smithi096.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T22:19:29.028 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T22:19:29.055 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:19:29.104 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:19:29.523 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T22:19:29.550 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:19:29.589 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:19:29.884 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T22:19:29.910 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:19:29.942 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:19:30.262 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T22:19:30.289 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T22:19:30.328 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:19:30.462 INFO:teuthology.orchestra.run.smithi096.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T22:19:30.681 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librados2. 2024-04-18T22:19:30.708 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:30.747 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:31.040 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 179 MB in 20s (9,027 kB/s) 2024-04-18T22:19:31.181 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T22:19:31.208 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librbd1. 2024-04-18T22:19:31.235 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:31.305 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T22:19:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T22:19:31.363 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:19:31.678 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T22:19:31.704 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:31.717 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T22:19:31.736 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:31.743 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T22:19:31.783 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:19:31.989 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2024-04-18T22:19:32.004 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:32.046 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:32.128 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T22:19:32.154 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T22:19:32.336 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:19:32.374 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T22:19:32.401 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:19:32.441 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:32.693 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T22:19:32.709 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:32.723 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T22:19:32.749 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:19:32.751 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:32.807 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:19:33.204 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T22:19:33.218 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T22:19:33.231 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:19:33.244 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:19:33.271 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:33.284 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:19:33.587 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T22:19:33.614 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:19:33.624 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T22:19:33.651 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T22:19:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:19:33.690 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:19:33.960 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T22:19:33.986 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T22:19:34.007 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T22:19:34.018 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T22:19:34.033 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T22:19:34.073 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:19:34.287 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T22:19:34.303 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:34.345 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:34.393 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librados2. 2024-04-18T22:19:34.419 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:34.459 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:34.698 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T22:19:34.725 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T22:19:34.773 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:19:34.887 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librbd1. 2024-04-18T22:19:34.913 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:34.953 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:35.059 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T22:19:35.086 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T22:19:35.125 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:19:35.247 INFO:teuthology.orchestra.run.smithi086.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T22:19:35.365 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T22:19:35.391 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:35.420 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T22:19:35.423 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:35.436 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T22:19:35.470 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:19:35.667 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rados. 2024-04-18T22:19:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:35.733 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:35.807 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua5.1. 2024-04-18T22:19:35.833 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T22:19:35.873 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:19:36.070 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T22:19:36.096 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:19:36.136 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:36.193 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua-any. 2024-04-18T22:19:36.219 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T22:19:36.259 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T22:19:36.389 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T22:19:36.403 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:36.495 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package zip. 2024-04-18T22:19:36.510 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T22:19:36.547 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T22:19:36.597 INFO:teuthology.orchestra.run.smithi196.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T22:19:36.766 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T22:19:36.793 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:19:36.833 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:36.866 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package unzip. 2024-04-18T22:19:36.893 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T22:19:36.933 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:19:37.119 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T22:19:37.135 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T22:19:37.169 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:19:37.328 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package luarocks. 2024-04-18T22:19:37.355 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T22:19:37.381 INFO:teuthology.orchestra.run.smithi086.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T22:19:37.395 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:19:37.466 INFO:teuthology.orchestra.run.smithi086.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T22:19:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T22:19:37.507 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T22:19:37.547 INFO:teuthology.orchestra.run.smithi086.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T22:19:37.703 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T22:19:37.752 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2024-04-18T22:19:37.779 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:37.819 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:37.983 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T22:19:37.998 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:38.049 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:38.104 INFO:teuthology.orchestra.run.smithi086.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T22:19:38.402 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T22:19:38.429 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T22:19:38.454 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T22:19:38.469 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:19:38.470 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:38.513 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:38.709 INFO:teuthology.orchestra.run.smithi196.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T22:19:38.738 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T22:19:38.761 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T22:19:38.771 INFO:teuthology.orchestra.run.smithi196.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T22:19:38.796 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:19:38.812 INFO:teuthology.orchestra.run.smithi196.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T22:19:38.874 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T22:19:38.901 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T22:19:38.940 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:19:39.099 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T22:19:39.115 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T22:19:39.157 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:19:39.194 INFO:teuthology.orchestra.run.smithi196.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T22:19:39.252 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T22:19:39.278 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:39.318 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:39.494 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua5.1. 2024-04-18T22:19:39.520 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T22:19:39.560 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:19:39.688 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2024-04-18T22:19:39.715 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:39.746 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:39.859 INFO:teuthology.orchestra.run.smithi086.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T22:19:39.872 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-any. 2024-04-18T22:19:39.887 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T22:19:39.922 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T22:19:40.141 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package zip. 2024-04-18T22:19:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T22:19:40.199 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T22:19:40.536 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package unzip. 2024-04-18T22:19:40.562 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T22:19:40.602 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:19:40.718 INFO:teuthology.orchestra.run.smithi196.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T22:19:41.014 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package luarocks. 2024-04-18T22:19:41.040 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T22:19:41.080 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:19:41.285 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2024-04-18T22:19:41.311 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:41.361 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:41.472 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librgw2. 2024-04-18T22:19:41.498 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:41.529 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:41.939 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T22:19:41.965 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T22:19:42.005 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:19:42.082 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T22:19:42.099 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:42.140 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:42.291 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T22:19:42.318 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T22:19:42.358 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:19:42.443 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T22:19:42.470 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T22:19:42.510 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:19:42.677 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T22:19:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T22:19:42.744 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:19:42.779 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T22:19:42.806 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:42.846 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:43.005 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T22:19:43.032 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T22:19:43.071 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:19:43.383 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T22:19:43.391 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-common. 2024-04-18T22:19:43.410 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T22:19:43.417 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:43.457 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:43.483 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:19:44.553 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T22:19:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T22:19:44.619 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T22:19:44.836 INFO:teuthology.orchestra.run.smithi086.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T22:19:44.837 INFO:teuthology.orchestra.run.smithi086.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T22:19:44.880 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-portend. 2024-04-18T22:19:44.896 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T22:19:44.930 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T22:19:44.933 INFO:teuthology.orchestra.run.smithi086.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T22:19:45.208 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T22:19:45.235 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T22:19:45.274 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T22:19:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T22:19:45.429 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-base. 2024-04-18T22:19:45.444 INFO:teuthology.orchestra.run.smithi086.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T22:19:45.455 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:45.474 INFO:teuthology.orchestra.run.smithi086.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T22:19:45.505 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:45.553 INFO:teuthology.orchestra.run.smithi196.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T22:19:45.554 INFO:teuthology.orchestra.run.smithi196.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T22:19:45.570 INFO:teuthology.orchestra.run.smithi196.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T22:19:45.577 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T22:19:45.604 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T22:19:45.643 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:19:45.861 INFO:teuthology.orchestra.run.smithi086.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T22:19:45.911 INFO:teuthology.orchestra.run.smithi196.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T22:19:45.955 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T22:19:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T22:19:45.990 INFO:teuthology.orchestra.run.smithi196.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T22:19:46.021 INFO:teuthology.orchestra.run.smithi196.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T22:19:46.022 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T22:19:46.059 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T22:19:46.085 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T22:19:46.125 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:19:46.289 INFO:teuthology.orchestra.run.smithi196.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T22:19:46.325 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T22:19:46.352 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T22:19:46.391 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T22:19:46.428 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T22:19:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T22:19:46.494 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:19:46.694 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T22:19:46.721 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T22:19:46.761 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:19:46.814 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T22:19:46.840 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T22:19:46.880 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:19:47.063 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2024-04-18T22:19:47.090 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T22:19:47.130 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:19:47.167 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T22:19:47.193 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T22:19:47.225 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:19:47.355 INFO:teuthology.orchestra.run.smithi196.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T22:19:47.403 INFO:teuthology.orchestra.run.smithi196.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T22:19:47.425 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T22:19:47.441 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T22:19:47.447 INFO:teuthology.orchestra.run.smithi086.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T22:19:47.448 INFO:teuthology.orchestra.run.smithi086.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T22:19:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:19:47.502 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T22:19:47.529 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T22:19:47.569 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:19:47.761 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T22:19:47.788 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T22:19:47.827 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:19:47.872 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T22:19:47.898 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T22:19:47.938 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T22:19:48.122 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2024-04-18T22:19:48.148 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T22:19:48.188 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:19:48.466 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-portend. 2024-04-18T22:19:48.492 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T22:19:48.533 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T22:19:48.791 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T22:19:48.802 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T22:19:48.818 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T22:19:48.828 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T22:19:48.853 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:19:48.860 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T22:19:49.127 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T22:19:49.154 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T22:19:49.180 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T22:19:49.193 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:19:49.206 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T22:19:49.246 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:19:49.480 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2024-04-18T22:19:49.506 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T22:19:49.546 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:19:49.592 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T22:19:49.618 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T22:19:49.658 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T22:19:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T22:19:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T22:19:49.899 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:19:49.928 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T22:19:49.955 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T22:19:49.986 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T22:19:50.202 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T22:19:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T22:19:50.268 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:19:50.305 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T22:19:50.332 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T22:19:50.372 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:19:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T22:19:50.598 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T22:19:50.638 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T22:19:50.675 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-mako. 2024-04-18T22:19:50.701 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T22:19:50.741 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:19:50.949 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T22:19:50.976 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T22:19:51.015 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:19:51.061 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T22:19:51.088 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T22:19:51.128 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:19:51.397 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T22:19:51.403 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T22:19:51.424 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T22:19:51.431 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T22:19:51.455 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:19:51.470 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T22:19:51.733 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webob. 2024-04-18T22:19:51.748 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-idna. 2024-04-18T22:19:51.760 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T22:19:51.764 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T22:19:51.797 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T22:19:51.800 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:19:52.117 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T22:19:52.133 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T22:19:52.175 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:19:52.178 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T22:19:52.204 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T22:19:52.247 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:19:52.495 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests. 2024-04-18T22:19:52.522 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T22:19:52.555 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T22:19:52.561 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:19:52.582 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T22:19:52.622 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:19:52.898 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T22:19:52.899 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-paste. 2024-04-18T22:19:52.915 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T22:19:52.925 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T22:19:52.950 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:19:52.964 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:19:53.259 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T22:19:53.275 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:19:53.286 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T22:19:53.313 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T22:19:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:53.353 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:19:53.672 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T22:19:53.680 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T22:19:53.699 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T22:19:53.707 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:53.738 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:19:54.042 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T22:19:54.049 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T22:19:54.069 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T22:19:54.077 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T22:19:54.108 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T22:19:54.116 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:19:54.386 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T22:19:54.412 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T22:19:54.419 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T22:19:54.444 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:19:54.446 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T22:19:54.485 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T22:19:54.801 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T22:19:54.813 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T22:19:54.828 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T22:19:54.840 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:54.867 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T22:19:54.879 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:55.204 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-idna. 2024-04-18T22:19:55.231 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T22:19:55.266 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T22:19:55.270 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T22:19:55.293 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:55.324 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:55.615 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T22:19:55.641 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T22:19:55.681 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:19:55.903 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T22:19:55.930 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:55.969 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:56.010 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests. 2024-04-18T22:19:56.036 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T22:19:56.076 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:19:56.387 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T22:19:56.414 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T22:19:56.454 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:19:56.723 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T22:19:56.749 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:19:56.781 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:57.064 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2024-04-18T22:19:57.080 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:57.122 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:57.186 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T22:19:57.213 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:57.253 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:57.417 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T22:19:57.443 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:57.483 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:57.513 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T22:19:57.540 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T22:19:57.572 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:19:57.819 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T22:19:57.846 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:57.850 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T22:19:57.876 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T22:19:57.885 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:57.916 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T22:19:57.993 INFO:teuthology.orchestra.run.smithi086.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T22:19:57.995 INFO:teuthology.orchestra.run.smithi086.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T22:19:57.996 INFO:teuthology.orchestra.run.smithi086.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T22:19:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T22:19:58.262 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:58.280 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2024-04-18T22:19:58.302 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:58.307 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:58.666 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T22:19:58.694 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T22:19:58.733 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T22:19:58.856 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T22:19:58.885 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:58.922 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:59.179 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T22:19:59.206 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:19:59.260 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:19:59.551 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T22:19:59.578 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:19:59.617 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:19:59.732 INFO:teuthology.orchestra.run.smithi196.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T22:19:59.734 INFO:teuthology.orchestra.run.smithi196.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T22:19:59.735 INFO:teuthology.orchestra.run.smithi196.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T22:20:00.298 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-babel. 2024-04-18T22:20:00.326 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:00.330 INFO:teuthology.orchestra.run.smithi086.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T22:20:00.364 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:00.662 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph. 2024-04-18T22:20:00.677 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:00.701 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T22:20:00.720 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:00.729 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T22:20:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:20:00.890 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 179 MB in 50s (3,596 kB/s) 2024-04-18T22:20:01.048 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T22:20:01.058 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T22:20:01.074 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:01.114 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T22:20:01.156 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:01.159 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T22:20:01.164 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T22:20:01.195 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:01.248 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:20:01.468 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T22:20:01.494 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:01.534 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:01.557 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T22:20:01.584 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T22:20:01.593 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T22:20:01.620 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T22:20:01.623 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:01.651 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:20:01.876 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2024-04-18T22:20:01.903 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T22:20:01.928 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephadm. 2024-04-18T22:20:01.934 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:20:01.955 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:02.065 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:02.065 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T22:20:02.065 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T22:20:02.065 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:20:02.229 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T22:20:02.256 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:02.295 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:02.314 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T22:20:02.341 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T22:20:02.380 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T22:20:02.496 INFO:teuthology.orchestra.run.smithi196.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T22:20:02.632 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T22:20:02.659 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:20:02.707 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:02.725 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T22:20:02.752 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:02.792 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:03.077 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T22:20:03.093 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:20:03.127 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:03.329 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 179 MB in 52s (3,421 kB/s) 2024-04-18T22:20:03.467 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T22:20:03.472 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T22:20:03.472 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T22:20:03.495 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T22:20:03.498 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:20:03.533 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:20:03.539 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:03.551 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T22:20:03.556 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T22:20:03.638 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:20:03.771 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-babel. 2024-04-18T22:20:03.787 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:03.821 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:03.958 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T22:20:03.984 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T22:20:04.044 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T22:20:04.071 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T22:20:04.112 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T22:20:04.116 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:20:04.128 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T22:20:04.145 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:20:04.161 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:20:04.420 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librados2. 2024-04-18T22:20:04.446 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:04.457 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T22:20:04.473 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T22:20:04.478 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:04.507 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:20:04.532 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T22:20:04.559 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T22:20:04.598 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:20:04.694 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T22:20:04.720 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T22:20:04.761 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:20:04.802 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T22:20:04.829 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:04.868 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:04.901 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T22:20:04.928 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T22:20:04.931 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librbd1. 2024-04-18T22:20:04.957 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:04.967 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:20:04.997 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:05.180 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T22:20:05.196 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T22:20:05.198 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T22:20:05.224 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:20:05.238 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:05.272 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:05.433 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T22:20:05.460 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:05.500 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:05.541 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-routes. 2024-04-18T22:20:05.568 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T22:20:05.608 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:20:05.634 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T22:20:05.660 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:20:05.717 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:05.786 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rados. 2024-04-18T22:20:05.812 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:05.842 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T22:20:05.852 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:05.870 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:05.944 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T22:20:05.972 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:06.002 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:06.010 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:06.045 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T22:20:06.071 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T22:20:06.111 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:06.147 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T22:20:06.173 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:06.205 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:06.424 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T22:20:06.451 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:06.465 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T22:20:06.491 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T22:20:06.491 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:06.531 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:20:06.810 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T22:20:06.837 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:06.876 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librados2. 2024-04-18T22:20:06.877 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:06.902 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:06.942 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:07.014 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T22:20:07.042 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T22:20:07.080 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T22:20:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T22:20:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T22:20:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:20:07.274 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T22:20:07.289 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T22:20:07.323 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:20:07.350 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T22:20:07.353 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librbd1. 2024-04-18T22:20:07.366 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T22:20:07.379 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:20:07.411 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:07.658 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T22:20:07.673 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T22:20:07.707 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T22:20:07.736 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T22:20:07.764 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T22:20:07.803 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:20:07.819 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T22:20:07.822 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T22:20:07.846 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T22:20:07.849 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:07.885 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:20:07.889 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:07.977 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T22:20:08.004 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:08.044 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:08.089 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T22:20:08.116 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rados. 2024-04-18T22:20:08.117 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T22:20:08.132 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:08.156 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T22:20:08.166 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:08.247 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T22:20:08.262 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T22:20:08.305 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:20:08.405 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T22:20:08.431 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T22:20:08.472 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:20:08.476 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T22:20:08.486 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T22:20:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T22:20:08.512 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T22:20:08.552 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:08.633 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T22:20:08.660 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T22:20:08.699 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:20:08.749 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T22:20:08.763 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T22:20:08.764 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T22:20:08.778 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:08.798 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:20:08.813 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:08.862 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T22:20:08.890 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T22:20:08.928 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:20:09.053 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T22:20:09.069 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T22:20:09.112 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:20:09.116 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T22:20:09.142 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:09.182 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:09.231 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T22:20:09.249 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T22:20:09.289 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T22:20:09.374 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T22:20:09.402 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:09.448 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua5.1. 2024-04-18T22:20:09.475 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T22:20:09.515 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:20:09.519 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T22:20:09.535 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:09.545 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T22:20:09.585 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:20:09.793 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T22:20:09.821 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T22:20:09.826 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-any. 2024-04-18T22:20:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T22:20:09.859 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:20:09.884 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T22:20:09.913 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T22:20:09.940 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T22:20:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T22:20:10.120 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package zip. 2024-04-18T22:20:10.147 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T22:20:10.187 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T22:20:10.224 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T22:20:10.239 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:10.274 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:10.505 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T22:20:10.506 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T22:20:10.506 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package unzip. 2024-04-18T22:20:10.521 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T22:20:10.523 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:10.533 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T22:20:10.554 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T22:20:10.564 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:10.573 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:20:10.627 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T22:20:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T22:20:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:20:10.824 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T22:20:10.834 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T22:20:10.850 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T22:20:10.851 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T22:20:10.883 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:20:10.899 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:20:10.945 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T22:20:10.961 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T22:20:10.968 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package luarocks. 2024-04-18T22:20:10.995 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T22:20:11.004 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:20:11.035 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:20:11.220 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T22:20:11.227 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T22:20:11.248 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:11.255 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T22:20:11.286 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:20:11.294 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:20:11.307 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T22:20:11.333 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T22:20:11.373 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:20:11.409 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librgw2. 2024-04-18T22:20:11.435 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:11.467 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:11.556 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2024-04-18T22:20:11.563 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T22:20:11.584 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:11.590 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T22:20:11.621 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T22:20:11.639 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T22:20:11.693 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua5.1. 2024-04-18T22:20:11.708 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T22:20:11.743 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:20:11.883 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2024-04-18T22:20:11.900 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T22:20:11.933 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:20:11.950 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T22:20:11.977 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T22:20:11.986 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T22:20:12.001 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:12.017 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T22:20:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:12.054 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-any. 2024-04-18T22:20:12.081 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T22:20:12.121 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T22:20:12.261 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T22:20:12.289 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T22:20:12.327 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:20:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T22:20:12.364 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T22:20:12.374 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package zip. 2024-04-18T22:20:12.389 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T22:20:12.397 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T22:20:12.403 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:20:12.424 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T22:20:12.432 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T22:20:12.463 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:20:12.698 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2024-04-18T22:20:12.714 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T22:20:12.714 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:12.741 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T22:20:12.752 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package unzip. 2024-04-18T22:20:12.756 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:12.778 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T22:20:12.780 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T22:20:12.818 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:20:12.966 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T22:20:12.993 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:13.033 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:13.084 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T22:20:13.111 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T22:20:13.142 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:20:13.206 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package luarocks. 2024-04-18T22:20:13.232 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T22:20:13.272 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:20:13.386 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-common. 2024-04-18T22:20:13.413 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:13.444 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:13.646 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librgw2. 2024-04-18T22:20:13.673 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:13.713 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:13.847 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T22:20:13.863 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:13.905 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:14.200 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T22:20:14.227 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T22:20:14.266 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:20:14.290 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T22:20:14.305 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:14.348 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:14.808 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-base. 2024-04-18T22:20:14.810 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T22:20:14.820 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T22:20:14.834 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:14.836 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T22:20:14.847 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:14.868 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:20:14.886 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:20:14.907 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:15.044 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T22:20:15.072 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:15.102 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:15.154 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T22:20:15.155 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package jq. 2024-04-18T22:20:15.171 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:15.180 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:15.213 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T22:20:15.220 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:15.389 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T22:20:15.406 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:15.447 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:15.466 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package socat. 2024-04-18T22:20:15.482 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T22:20:15.503 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T22:20:15.524 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:20:15.530 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T22:20:15.569 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:20:15.641 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-common. 2024-04-18T22:20:15.667 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:15.707 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:15.809 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T22:20:15.836 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T22:20:15.847 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T22:20:15.861 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T22:20:15.874 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T22:20:15.875 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:20:15.877 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T22:20:15.914 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:20:15.919 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:20:16.209 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T22:20:16.212 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T22:20:16.235 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T22:20:16.240 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T22:20:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-test. 2024-04-18T22:20:16.267 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:20:16.273 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:16.279 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:20:16.314 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:16.486 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T22:20:16.512 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T22:20:16.552 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:20:16.574 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T22:20:16.603 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T22:20:16.641 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:16.855 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T22:20:16.882 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T22:20:16.922 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:20:17.037 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-base. 2024-04-18T22:20:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package pkg-config. 2024-04-18T22:20:17.063 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:17.076 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T22:20:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:20:17.138 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:17.208 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T22:20:17.234 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T22:20:17.266 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T22:20:17.375 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T22:20:17.403 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T22:20:17.442 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:17.527 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-portend. 2024-04-18T22:20:17.553 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T22:20:17.593 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T22:20:17.683 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T22:20:17.709 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T22:20:17.741 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:20:17.895 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T22:20:17.905 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T22:20:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T22:20:17.931 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T22:20:17.961 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:17.971 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T22:20:18.044 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T22:20:18.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T22:20:18.144 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:20:18.206 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T22:20:18.223 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T22:20:18.255 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T22:20:18.291 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T22:20:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T22:20:18.357 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:20:18.472 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T22:20:18.498 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T22:20:18.509 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T22:20:18.522 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T22:20:18.538 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:20:18.550 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:20:18.561 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T22:20:18.588 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:18.619 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:18.702 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T22:20:18.729 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T22:20:18.769 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T22:20:18.824 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T22:20:18.851 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T22:20:18.862 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T22:20:18.890 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T22:20:18.891 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:20:18.905 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T22:20:18.922 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:18.928 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T22:20:18.964 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:19.064 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T22:20:19.090 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T22:20:19.130 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T22:20:19.160 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T22:20:19.176 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T22:20:19.210 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:20:19.258 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T22:20:19.274 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T22:20:19.281 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2024-04-18T22:20:19.308 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:20:19.309 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T22:20:19.348 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T22:20:19.432 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T22:20:19.459 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T22:20:19.496 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T22:20:19.499 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:20:19.522 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T22:20:19.563 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T22:20:19.587 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T22:20:19.603 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T22:20:19.643 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T22:20:19.645 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:20:19.671 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T22:20:19.709 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:20:19.743 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-mako. 2024-04-18T22:20:19.770 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T22:20:19.810 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:20:19.876 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-portend. 2024-04-18T22:20:19.902 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T22:20:19.943 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T22:20:20.033 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T22:20:20.052 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T22:20:20.091 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:20.138 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T22:20:20.164 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T22:20:20.205 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:20:20.213 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T22:20:20.241 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T22:20:20.245 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T22:20:20.272 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T22:20:20.280 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:20:20.312 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T22:20:20.507 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T22:20:20.522 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T22:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package pkg-config. 2024-04-18T22:20:20.556 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T22:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T22:20:20.565 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:20:20.582 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T22:20:20.583 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-toml. 2024-04-18T22:20:20.611 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T22:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:20:20.614 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:20:20.649 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T22:20:20.818 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webob. 2024-04-18T22:20:20.844 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T22:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T22:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T22:20:20.876 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:20:20.901 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T22:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:20.918 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T22:20:20.951 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:20:20.960 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T22:20:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T22:20:21.026 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T22:20:21.212 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T22:20:21.239 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T22:20:21.282 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:20:21.288 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T22:20:21.317 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T22:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T22:20:21.337 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T22:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T22:20:21.355 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:20:21.364 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T22:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:21.404 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T22:20:21.615 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T22:20:21.616 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T22:20:21.642 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T22:20:21.644 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T22:20:21.674 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:20:21.679 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T22:20:21.682 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:20:21.706 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T22:20:21.707 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T22:20:21.733 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T22:20:21.745 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T22:20:21.774 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:20:21.985 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-paste. 2024-04-18T22:20:22.011 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T22:20:22.051 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:20:22.051 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-mako. 2024-04-18T22:20:22.065 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T22:20:22.066 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T22:20:22.093 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T22:20:22.101 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:20:22.132 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:20:22.362 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T22:20:22.377 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T22:20:22.411 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2024-04-18T22:20:22.412 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T22:20:22.420 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:20:22.428 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T22:20:22.440 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:22.443 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T22:20:22.471 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:20:22.471 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T22:20:22.478 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:22.510 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T22:20:22.698 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T22:20:22.725 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T22:20:22.756 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:20:22.782 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T22:20:22.808 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T22:20:22.848 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:20:22.854 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-py. 2024-04-18T22:20:22.882 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T22:20:22.929 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T22:20:23.151 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webob. 2024-04-18T22:20:23.163 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T22:20:23.177 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T22:20:23.189 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T22:20:23.218 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:20:23.249 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T22:20:23.251 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T22:20:23.276 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T22:20:23.316 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:20:23.514 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T22:20:23.531 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:23.545 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T22:20:23.562 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T22:20:23.572 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T22:20:23.572 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:23.588 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T22:20:23.611 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:20:23.631 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:20:23.795 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T22:20:23.822 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T22:20:23.861 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:20:23.877 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:23.940 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T22:20:23.966 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T22:20:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T22:20:23.993 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T22:20:24.006 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:20:24.033 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T22:20:24.144 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:20:24.197 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-toml. 2024-04-18T22:20:24.224 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T22:20:24.263 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T22:20:24.270 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:24.292 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-paste. 2024-04-18T22:20:24.319 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T22:20:24.359 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:20:24.360 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-idna. 2024-04-18T22:20:24.387 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T22:20:24.427 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T22:20:24.544 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T22:20:24.578 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T22:20:24.603 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T22:20:24.641 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:20:24.687 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T22:20:24.696 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T22:20:24.702 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T22:20:24.723 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T22:20:24.745 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:20:24.754 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:20:24.842 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T22:20:24.995 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T22:20:25.022 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T22:20:25.048 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T22:20:25.061 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:20:25.074 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T22:20:25.077 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests. 2024-04-18T22:20:25.101 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T22:20:25.114 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:20:25.141 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:20:25.247 INFO:teuthology.orchestra.run.smithi032.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T22:20:25.313 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T22:20:25.322 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:25.329 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T22:20:25.372 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:20:25.417 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T22:20:25.444 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T22:20:25.444 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T22:20:25.470 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T22:20:25.475 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T22:20:25.510 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:20:25.753 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T22:20:25.780 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T22:20:25.805 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T22:20:25.831 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:25.836 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:20:25.871 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:25.905 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2024-04-18T22:20:26.061 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:20:26.109 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package radosgw. 2024-04-18T22:20:26.137 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:26.174 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T22:20:26.175 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:26.190 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T22:20:26.233 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T22:20:26.251 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T22:20:26.277 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:26.317 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:26.329 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:20:26.502 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-idna. 2024-04-18T22:20:26.529 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T22:20:26.569 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T22:20:26.572 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:26.628 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T22:20:26.655 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T22:20:26.695 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:20:26.698 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:20:26.880 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T22:20:26.906 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T22:20:26.946 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:20:26.969 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T22:20:26.994 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:26.997 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T22:20:27.007 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:27.024 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T22:20:27.028 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T22:20:27.266 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests. 2024-04-18T22:20:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:20:27.293 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T22:20:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:27.332 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:20:27.350 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T22:20:27.377 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:27.393 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T22:20:27.408 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:27.591 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:20:27.644 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T22:20:27.670 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T22:20:27.703 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:20:27.710 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:20:27.742 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:27.803 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T22:20:27.829 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:27.869 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:27.996 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T22:20:28.005 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T22:20:28.012 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:28.054 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:28.088 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T22:20:28.331 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T22:20:28.339 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T22:20:28.409 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T22:20:28.436 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:28.468 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:20:28.665 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T22:20:28.691 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:28.731 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:28.747 INFO:teuthology.orchestra.run.smithi096.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T22:20:28.770 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T22:20:28.796 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T22:20:28.819 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:28.837 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:20:28.842 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:20:29.101 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:20:29.139 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T22:20:29.166 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T22:20:29.206 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T22:20:29.327 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user cephadm....done 2024-04-18T22:20:29.377 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:20:29.483 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:20:29.551 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T22:20:29.577 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:29.628 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:29.653 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:29.743 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:20:29.784 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph. 2024-04-18T22:20:29.811 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:29.842 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:29.871 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:20:29.972 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:20:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:30.073 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:20:30.103 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:20:30.145 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T22:20:30.172 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:30.199 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:20:30.211 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:30.245 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T22:20:30.272 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:30.312 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:30.362 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:30.467 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:20:30.573 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T22:20:30.599 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:30.622 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:20:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:30.748 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T22:20:30.819 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T22:20:30.857 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T22:20:30.882 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T22:20:30.904 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:20:30.908 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:30.948 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:31.013 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T22:20:31.034 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephadm. 2024-04-18T22:20:31.050 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:20:31.060 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:31.100 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:31.381 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:31.386 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T22:20:31.413 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T22:20:31.427 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T22:20:31.444 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T22:20:31.490 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T22:20:31.653 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T22:20:31.758 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:20:31.797 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T22:20:31.823 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:31.863 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:31.912 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:20:31.960 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph. 2024-04-18T22:20:31.975 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:32.018 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:32.026 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:20:32.147 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:20:32.337 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T22:20:32.364 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:32.369 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T22:20:32.403 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:20:32.470 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:20:32.674 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:20:32.760 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T22:20:32.760 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:32.790 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:32.926 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-babel. 2024-04-18T22:20:32.933 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:32.953 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:32.993 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:33.164 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:20:33.168 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephadm. 2024-04-18T22:20:33.194 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:33.234 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:33.243 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:20:33.321 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T22:20:33.344 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:20:33.347 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T22:20:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:20:33.445 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:20:33.448 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:20:33.554 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:20:33.562 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T22:20:33.574 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:20:33.589 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T22:20:33.628 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T22:20:33.700 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:20:33.732 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T22:20:33.759 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:33.798 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:33.813 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:20:33.922 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T22:20:33.946 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:33.989 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:34.009 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T22:20:34.135 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T22:20:34.161 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T22:20:34.194 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T22:20:34.201 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:34.229 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T22:20:34.292 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:20:34.401 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T22:20:34.403 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T22:20:34.504 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-routes. 2024-04-18T22:20:34.530 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T22:20:34.570 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:20:34.637 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:20:34.735 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:34.836 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T22:20:34.924 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T22:20:34.950 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:34.952 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-babel. 2024-04-18T22:20:34.979 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T22:20:34.990 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:35.005 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:20:35.010 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:35.112 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:20:35.298 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:20:35.463 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T22:20:35.490 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T22:20:35.530 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:20:35.666 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:20:35.730 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:20:35.841 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T22:20:35.856 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:35.899 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:35.918 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:20:36.042 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T22:20:36.043 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:20:36.194 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T22:20:36.209 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T22:20:36.244 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:36.312 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T22:20:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T22:20:36.345 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T22:20:36.361 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:20:36.547 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-routes. 2024-04-18T22:20:36.573 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T22:20:36.613 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:20:36.655 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:20:36.770 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:20:36.823 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T22:20:36.850 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T22:20:36.881 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:20:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T22:20:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:37.037 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:20:37.163 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:20:37.218 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T22:20:37.245 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T22:20:37.284 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:20:37.289 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:20:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T22:20:37.587 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T22:20:37.614 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T22:20:37.653 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:20:37.810 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:20:37.975 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T22:20:38.037 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:38.146 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T22:20:38.202 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:20:38.271 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T22:20:38.287 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T22:20:38.329 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:20:38.345 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T22:20:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:38.422 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:20:38.505 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:38.549 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:20:38.561 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:20:38.674 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:20:38.816 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T22:20:38.843 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T22:20:38.846 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:20:38.874 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:20:38.976 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:20:39.189 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:20:39.236 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T22:20:39.243 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T22:20:39.263 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T22:20:39.302 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:20:39.440 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:20:39.442 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T22:20:39.469 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T22:20:39.508 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T22:20:39.511 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:39.566 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:39.605 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T22:20:39.612 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:20:39.631 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T22:20:39.688 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:20:39.828 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T22:20:39.855 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T22:20:39.867 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T22:20:39.889 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:20:39.911 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:20:40.148 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T22:20:40.160 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:20:40.256 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T22:20:40.283 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T22:20:40.331 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:20:40.363 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T22:20:40.391 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:40.416 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T22:20:40.506 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:40.642 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T22:20:40.669 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T22:20:40.684 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:20:40.708 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T22:20:40.952 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T22:20:40.970 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T22:20:40.985 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T22:20:41.020 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T22:20:41.203 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:20:41.312 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T22:20:41.323 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T22:20:41.325 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:20:41.350 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T22:20:41.389 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:20:41.451 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T22:20:41.479 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T22:20:41.518 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T22:20:41.568 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:41.580 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:20:41.677 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T22:20:41.706 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:20:41.734 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T22:20:41.761 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T22:20:41.770 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T22:20:41.794 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T22:20:41.800 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T22:20:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:20:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:20:42.007 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:20:42.079 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:20:42.112 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T22:20:42.123 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T22:20:42.140 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T22:20:42.151 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T22:20:42.178 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:20:42.190 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:20:42.205 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:20:42.308 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:20:42.460 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T22:20:42.487 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T22:20:42.498 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:20:42.526 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T22:20:42.559 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:20:42.685 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:20:42.758 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T22:20:42.812 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:42.838 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T22:20:42.865 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T22:20:42.875 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T22:20:42.883 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T22:20:42.903 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:42.904 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T22:20:42.942 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:42.992 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:43.094 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:20:43.130 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:20:43.231 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:20:43.233 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T22:20:43.260 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T22:20:43.269 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T22:20:43.296 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T22:20:43.299 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:20:43.336 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:20:43.387 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:20:43.610 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T22:20:43.616 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:20:43.638 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T22:20:43.646 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T22:20:43.677 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T22:20:43.680 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T22:20:43.708 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:20:43.885 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T22:20:43.906 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T22:20:43.972 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T22:20:43.983 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package jq. 2024-04-18T22:20:43.999 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:43.999 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T22:20:44.009 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T22:20:44.030 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:20:44.165 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:20:44.179 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T22:20:44.261 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:44.412 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:20:44.416 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T22:20:44.510 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package socat. 2024-04-18T22:20:44.526 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T22:20:44.568 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:20:44.627 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T22:20:44.643 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:44.675 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:20:44.685 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:20:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T22:20:44.888 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T22:20:44.904 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T22:20:44.938 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:20:45.038 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T22:20:45.039 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:20:45.066 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T22:20:45.069 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:20:45.104 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:20:45.196 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:20:45.299 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T22:20:45.300 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-test. 2024-04-18T22:20:45.316 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:45.358 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:45.424 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T22:20:45.441 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T22:20:45.468 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:45.488 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:20:45.507 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:20:45.692 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T22:20:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:20:45.905 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package jq. 2024-04-18T22:20:45.933 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T22:20:45.943 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:20:46.001 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T22:20:46.041 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:20:46.159 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:20:46.229 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package socat. 2024-04-18T22:20:46.245 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T22:20:46.268 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:20:46.287 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:20:46.297 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:20:46.339 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T22:20:46.595 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:20:46.615 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T22:20:46.631 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T22:20:46.673 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:20:46.673 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T22:20:46.729 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:20:46.967 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:20:47.036 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-test. 2024-04-18T22:20:47.063 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:47.102 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:47.122 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:20:47.311 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:20:47.382 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T22:20:47.445 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:47.500 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T22:20:47.512 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T22:20:47.539 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:47.579 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:47.579 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:47.706 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:20:47.759 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:20:47.860 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:20:47.883 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T22:20:47.910 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:47.949 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:48.058 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:48.169 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:20:48.209 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:48.277 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T22:20:48.304 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T22:20:48.343 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:20:48.360 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T22:20:48.471 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:20:48.631 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T22:20:48.658 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T22:20:48.687 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:48.688 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:20:48.747 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T22:20:48.813 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:48.873 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T22:20:48.939 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:49.061 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T22:20:49.089 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T22:20:49.128 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:49.149 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T22:20:49.170 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:20:49.170 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:20:49.231 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T22:20:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T22:20:49.298 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:49.442 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:20:49.591 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pkg-config. 2024-04-18T22:20:49.610 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T22:20:49.618 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T22:20:49.637 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:49.649 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:20:49.676 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:49.682 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:49.778 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:20:49.808 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T22:20:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T22:20:49.917 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:49.987 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T22:20:50.015 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T22:20:50.018 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:20:50.120 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:20:50.172 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T22:20:50.329 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T22:20:50.345 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T22:20:50.379 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:50.425 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T22:20:50.452 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T22:20:50.457 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:20:50.483 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:20:50.632 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T22:20:50.659 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T22:20:50.698 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T22:20:50.793 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:20:50.845 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T22:20:50.872 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T22:20:50.911 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:50.911 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:51.019 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T22:20:51.046 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T22:20:51.085 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:20:51.220 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.272 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.317 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pkg-config. 2024-04-18T22:20:51.344 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T22:20:51.346 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:20:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:20:51.396 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T22:20:51.398 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.423 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T22:20:51.487 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T22:20:51.533 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.644 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T22:20:51.660 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T22:20:51.683 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.683 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.702 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:51.798 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-py. 2024-04-18T22:20:51.800 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.826 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T22:20:51.834 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:51.865 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T22:20:51.901 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T22:20:52.105 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T22:20:52.121 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T22:20:52.155 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:52.161 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:52.193 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T22:20:52.221 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T22:20:52.260 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:20:52.288 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:52.414 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:52.436 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2024-04-18T22:20:52.458 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T22:20:52.485 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T22:20:52.524 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T22:20:52.626 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:20:52.626 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:20:52.747 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T22:20:52.774 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T22:20:52.812 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2024-04-18T22:20:52.813 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:20:52.878 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T22:20:52.905 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T22:20:52.944 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:20:53.032 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2024-04-18T22:20:53.047 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T22:20:53.082 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:53.090 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-toml. 2024-04-18T22:20:53.118 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T22:20:53.149 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T22:20:53.193 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T22:20:53.208 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:53.255 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T22:20:53.283 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T22:20:53.317 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:53.321 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T22:20:53.435 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:20:53.435 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T22:20:53.463 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T22:20:53.500 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T22:20:53.501 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:20:53.666 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-py. 2024-04-18T22:20:53.694 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T22:20:53.733 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T22:20:53.805 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T22:20:53.833 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T22:20:53.863 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:20:53.995 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:54.053 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T22:20:54.080 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T22:20:54.110 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T22:20:54.119 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:20:54.123 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T22:20:54.138 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:54.157 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:20:54.548 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T22:20:54.564 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T22:20:54.606 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:20:54.747 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:54.873 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:54.884 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-toml. 2024-04-18T22:20:54.911 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T22:20:54.927 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package radosgw. 2024-04-18T22:20:54.950 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T22:20:54.955 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:54.993 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:55.007 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:55.048 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:20:55.048 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:20:55.117 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:55.218 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:55.253 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T22:20:55.281 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T22:20:55.319 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:20:55.490 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:55.631 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T22:20:55.659 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T22:20:55.697 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:20:55.753 INFO:teuthology.orchestra.run.smithi096.stdout:Adding group ceph....done 2024-04-18T22:20:55.788 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T22:20:55.816 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:55.855 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:55.961 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T22:20:55.976 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T22:20:56.019 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:20:56.087 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user ceph....done 2024-04-18T22:20:56.138 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T22:20:56.159 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:56.239 INFO:teuthology.orchestra.run.smithi096.stdout:Setting system user ceph properties....done 2024-04-18T22:20:56.254 INFO:teuthology.orchestra.run.smithi096.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T22:20:56.387 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T22:20:56.409 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:20:56.527 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:56.609 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:56.687 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T22:20:56.706 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package radosgw. 2024-04-18T22:20:56.734 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:56.764 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:56.793 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T22:20:56.878 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:20:56.878 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:20:57.161 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:57.167 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T22:20:57.287 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:57.319 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:20:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:20:57.583 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T22:20:57.596 INFO:teuthology.orchestra.run.smithi086.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T22:20:57.611 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T22:20:57.650 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:57.662 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:57.752 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:20:57.752 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:20:57.937 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T22:20:58.054 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:58.136 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user cephadm....done 2024-04-18T22:20:58.204 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T22:20:58.222 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:58.284 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:20:58.330 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T22:20:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:20:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:20:58.563 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T22:20:58.569 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:20:58.597 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T22:20:58.820 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:20:58.822 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:58.938 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T22:20:58.938 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:20:58.948 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:59.069 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:59.099 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:59.164 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T22:20:59.300 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:20:59.301 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:20:59.352 INFO:teuthology.orchestra.run.smithi196.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T22:20:59.356 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:20:59.356 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:20:59.398 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:20:59.423 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:59.524 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T22:20:59.820 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:59.889 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user cephadm....done 2024-04-18T22:20:59.916 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:20:59.918 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:21:00.029 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:21:00.030 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:21:00.037 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T22:21:00.085 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:00.245 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:00.278 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T22:21:00.305 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T22:21:00.421 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:00.479 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:00.529 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T22:21:00.590 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T22:21:00.716 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T22:21:00.755 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:21:00.755 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:21:00.790 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T22:21:00.796 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:21:00.908 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:00.975 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T22:21:01.014 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:21:01.094 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:01.231 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:01.235 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T22:21:01.249 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:21:01.357 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:01.361 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T22:21:01.491 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:01.500 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:21:01.695 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T22:21:01.717 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:21:01.717 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:21:01.759 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:02.047 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T22:21:02.102 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:21:02.193 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:02.219 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:02.228 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:21:02.315 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T22:21:02.344 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:21:02.470 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:02.472 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:21:02.591 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T22:21:02.621 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:02.706 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:21:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:02.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-04-18T22:21:02.918 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T22:21:02.975 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T22:21:03.018 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-18T22:21:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:03.079 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T22:21:03.113 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T22:21:03.152 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T22:21:03.176 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:21:03.211 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:03.224 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-04-18T22:21:03.224 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-04-18T22:21:03.302 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T22:21:03.412 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T22:21:03.423 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-18T22:21:03.423 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-18T22:21:03.423 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:21:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:21:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:21:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:21:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:21:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:21:03.424 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:21:03.424 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-04-18T22:21:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: libfl-dev libfl2 libssl3 libuuid1 2024-04-18T22:21:03.427 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-04-18T22:21:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: bison-doc flex-doc libssl-doc 2024-04-18T22:21:03.460 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-04-18T22:21:03.461 INFO:teuthology.orchestra.run.smithi032.stdout: bison flex libaio-dev libelf-dev libfl-dev libfl2 libssl-dev 2024-04-18T22:21:03.463 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2024-04-18T22:21:03.463 INFO:teuthology.orchestra.run.smithi032.stdout: libssl3 libuuid1 uuid-dev 2024-04-18T22:21:03.653 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:21:03.654 INFO:teuthology.orchestra.run.smithi032.stdout:3 upgraded, 7 newly installed, 0 to remove and 299 not upgraded. 2024-04-18T22:21:03.654 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 5,495 kB of archives. 2024-04-18T22:21:03.654 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 16.5 MB of additional disk space will be used. 2024-04-18T22:21:03.654 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-18T22:21:03.754 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T22:21:03.754 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:04.005 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:21:04.020 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:04.130 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T22:21:04.137 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-18T22:21:04.256 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:21:04.290 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T22:21:04.395 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-18T22:21:04.396 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-18T22:21:04.397 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-18T22:21:04.416 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T22:21:04.431 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-18T22:21:04.432 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-18T22:21:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-18T22:21:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-18T22:21:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-18T22:21:04.533 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T22:21:04.540 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T22:21:04.650 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:21:04.689 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:04.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-04-18T22:21:04.776 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T22:21:04.792 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-04-18T22:21:04.863 INFO:teuthology.orchestra.run.smithi032.stdout:Preconfiguring packages ... 2024-04-18T22:21:04.966 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 5,495 kB in 1s (5,288 kB/s) 2024-04-18T22:21:04.993 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-04-18T22:21:04.994 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-04-18T22:21:05.171 INFO:teuthology.orchestra.run.smithi096.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-18T22:21:05.171 INFO:teuthology.orchestra.run.smithi096.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-18T22:21:05.171 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:21:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:21:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:21:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:21:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:21:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:21:05.173 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:21:05.173 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-04-18T22:21:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: libfl-dev libfl2 libssl3 libuuid1 2024-04-18T22:21:05.175 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-04-18T22:21:05.175 INFO:teuthology.orchestra.run.smithi096.stdout: bison-doc flex-doc libssl-doc 2024-04-18T22:21:05.203 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-04-18T22:21:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: bison flex libaio-dev libelf-dev libfl-dev libfl2 libssl-dev 2024-04-18T22:21:05.205 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-04-18T22:21:05.205 INFO:teuthology.orchestra.run.smithi096.stdout: libssl3 libuuid1 uuid-dev 2024-04-18T22:21:05.225 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package flex. 2024-04-18T22:21:05.253 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T22:21:05.282 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T22:21:05.292 INFO:teuthology.orchestra.run.smithi096.stdout:3 upgraded, 7 newly installed, 0 to remove and 299 not upgraded. 2024-04-18T22:21:05.292 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 5,495 kB of archives. 2024-04-18T22:21:05.292 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 16.5 MB of additional disk space will be used. 2024-04-18T22:21:05.292 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-18T22:21:05.327 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:21:05.333 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173891 files and directories currently installed.) 2024-04-18T22:21:05.339 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:05.346 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T22:21:05.390 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-18T22:21:05.416 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-18T22:21:05.472 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T22:21:05.496 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-18T22:21:05.498 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-18T22:21:05.498 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-18T22:21:05.514 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-18T22:21:05.514 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-18T22:21:05.516 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-18T22:21:05.516 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-18T22:21:05.517 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-18T22:21:05.569 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:21:05.670 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:21:05.771 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:21:05.848 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T22:21:05.876 INFO:teuthology.orchestra.run.smithi096.stdout:Preconfiguring packages ... 2024-04-18T22:21:05.932 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:05.974 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T22:21:05.983 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 5,495 kB in 0s (16.4 MB/s) 2024-04-18T22:21:06.071 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-18T22:21:06.080 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T22:21:06.117 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package flex. 2024-04-18T22:21:06.208 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173891 files and directories currently installed.) 2024-04-18T22:21:06.214 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:06.233 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T22:21:06.249 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-18T22:21:06.457 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T22:21:06.542 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:06.593 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T22:21:06.691 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:21:07.072 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:07.076 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:21:07.077 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:07.186 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T22:21:07.200 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:07.229 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T22:21:07.369 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:21:07.390 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-18T22:21:07.695 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:21:07.697 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:07.801 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:07.852 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:07.938 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:21:08.005 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T22:21:08.064 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:08.222 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:08.298 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T22:21:08.309 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:08.314 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:08.398 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T22:21:08.415 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T22:21:08.448 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:08.534 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T22:21:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package bison. 2024-04-18T22:21:08.699 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:21:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-18T22:21:08.771 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:08.835 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T22:21:08.865 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:08.996 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:09.108 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-18T22:21:09.124 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-18T22:21:09.166 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:09.205 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T22:21:09.344 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:09.454 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T22:21:09.470 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-18T22:21:09.498 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-18T22:21:09.545 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:09.706 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package bison. 2024-04-18T22:21:09.810 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:09.816 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-18T22:21:09.830 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T22:21:09.866 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:09.896 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:21:09.907 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-18T22:21:09.935 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:09.973 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:10.131 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T22:21:10.139 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:21:10.233 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-18T22:21:10.234 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-18T22:21:10.250 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-18T22:21:10.251 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:10.273 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T22:21:10.292 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:10.300 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:10.474 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T22:21:10.520 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-18T22:21:10.533 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:21:10.537 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-18T22:21:10.579 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:10.603 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-18T22:21:10.632 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:10.667 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:21:10.678 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T22:21:10.793 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:21:10.843 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:10.949 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-18T22:21:10.977 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:11.016 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:11.078 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:21:11.136 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T22:21:11.157 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-18T22:21:11.251 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-18T22:21:11.267 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:11.301 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:11.329 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T22:21:11.369 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:11.454 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T22:21:11.495 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:11.505 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-18T22:21:11.520 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:11.554 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:11.572 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:21:11.621 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:11.698 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:21:11.755 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:11.881 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:11.999 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:21:12.007 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:12.030 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-18T22:21:12.089 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-18T22:21:12.167 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:12.210 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:12.250 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T22:21:12.269 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:12.412 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:12.463 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:12.518 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T22:21:12.589 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:12.626 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T22:21:12.715 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:12.769 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:21:12.841 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:12.911 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:21:12.929 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-18T22:21:12.992 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:13.012 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T22:21:13.045 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T22:21:13.102 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:13.254 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:13.279 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:21:13.305 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T22:21:13.397 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T22:21:13.431 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T22:21:13.557 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T22:21:13.681 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:21:13.722 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:21:13.842 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T22:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:21:14.093 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T22:21:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:14.268 DEBUG:teuthology.parallel:result is None 2024-04-18T22:21:14.327 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:21:14.453 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T22:21:14.522 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:21:14.755 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T22:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T22:21:15.005 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T22:21:15.018 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:15.022 DEBUG:teuthology.parallel:result is None 2024-04-18T22:21:15.139 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:21:15.240 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:21:15.265 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T22:21:15.508 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T22:21:15.616 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:21:15.751 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T22:21:15.893 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T22:21:16.002 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T22:21:16.019 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T22:21:16.128 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T22:21:16.270 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:21:16.370 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:21:16.421 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T22:21:16.547 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T22:21:16.663 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:21:16.839 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T22:21:16.972 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:21:17.132 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T22:21:17.240 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T22:21:17.358 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T22:21:17.392 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T22:21:17.493 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T22:21:17.619 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T22:21:17.634 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T22:21:17.707 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T22:21:17.918 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:21:17.937 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T22:21:18.063 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T22:21:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:21:18.448 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T22:21:18.614 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T22:21:18.707 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T22:21:18.833 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T22:21:18.899 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:21:19.084 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T22:21:19.210 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T22:21:19.235 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:21:19.344 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:19.454 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:19.520 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T22:21:19.554 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:21:19.837 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T22:21:19.891 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:20.016 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:20.114 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T22:21:20.176 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T22:21:20.240 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T22:21:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:20.516 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T22:21:20.562 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:20.663 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:20.767 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T22:21:20.866 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:21:20.866 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:21:21.103 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T22:21:21.347 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:21.480 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T22:21:21.498 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:21.624 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:21.750 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:21:21.773 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T22:21:22.134 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T22:21:22.260 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:22.386 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:22.512 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T22:21:22.848 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:22.990 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:22.995 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.096 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.100 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T22:21:23.197 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.298 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.398 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.407 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.653 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:23.864 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:21:23.864 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T22:21:23.967 INFO:teuthology.orchestra.run.smithi086.stdout:Adding group ceph....done 2024-04-18T22:21:24.318 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user ceph....done 2024-04-18T22:21:24.338 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:24.464 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:24.470 INFO:teuthology.orchestra.run.smithi086.stdout:Setting system user ceph properties....done 2024-04-18T22:21:24.488 INFO:teuthology.orchestra.run.smithi086.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T22:21:24.590 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:24.624 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T22:21:24.724 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T22:21:24.978 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T22:21:25.491 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:25.626 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:25.928 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:26.054 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:26.117 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:21:26.117 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:21:26.180 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:26.306 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:26.432 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:26.603 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:26.950 INFO:teuthology.orchestra.run.smithi196.stdout:Adding group ceph....done 2024-04-18T22:21:26.987 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T22:21:27.376 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user ceph....done 2024-04-18T22:21:27.497 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:27.536 INFO:teuthology.orchestra.run.smithi196.stdout:Setting system user ceph properties....done 2024-04-18T22:21:27.552 INFO:teuthology.orchestra.run.smithi196.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T22:21:27.690 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T22:21:27.743 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:21:27.743 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:21:28.027 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T22:21:28.257 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:28.476 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:21:28.476 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:21:28.776 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:28.910 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:29.233 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:21:29.234 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:21:29.369 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:21:29.369 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T22:21:29.718 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:29.804 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:29.819 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:29.945 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:30.138 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:21:30.138 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:21:30.153 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T22:21:30.605 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:30.697 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:30.809 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:21:30.809 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T22:21:30.848 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:31.159 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:31.240 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:31.439 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:21:31.440 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T22:21:31.636 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T22:21:31.737 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:31.881 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:31.892 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:32.174 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:21:32.174 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T22:21:32.871 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:32.948 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:33.037 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:33.188 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:33.399 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:21:33.399 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T22:21:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:33.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-04-18T22:21:33.661 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-18T22:21:33.860 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-18T22:21:33.860 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-18T22:21:33.864 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:33.998 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:34.036 INFO:teuthology.orchestra.run.smithi086.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-18T22:21:34.036 INFO:teuthology.orchestra.run.smithi086.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-18T22:21:34.036 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:21:34.036 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:21:34.036 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:21:34.037 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:21:34.037 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:21:34.037 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:21:34.037 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:21:34.038 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-18T22:21:34.038 INFO:teuthology.orchestra.run.smithi086.stdout: libfl-dev libfl2 libssl3 libuuid1 2024-04-18T22:21:34.039 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-18T22:21:34.040 INFO:teuthology.orchestra.run.smithi086.stdout: bison-doc flex-doc libssl-doc 2024-04-18T22:21:34.068 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-18T22:21:34.069 INFO:teuthology.orchestra.run.smithi086.stdout: bison flex libaio-dev libelf-dev libfl-dev libfl2 libssl-dev 2024-04-18T22:21:34.070 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2024-04-18T22:21:34.071 INFO:teuthology.orchestra.run.smithi086.stdout: libssl3 libuuid1 uuid-dev 2024-04-18T22:21:34.124 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:34.251 INFO:teuthology.orchestra.run.smithi086.stdout:3 upgraded, 7 newly installed, 0 to remove and 299 not upgraded. 2024-04-18T22:21:34.251 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 5,495 kB of archives. 2024-04-18T22:21:34.251 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 16.5 MB of additional disk space will be used. 2024-04-18T22:21:34.251 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-18T22:21:34.275 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T22:21:34.620 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T22:21:34.729 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:34.735 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-18T22:21:34.865 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:34.992 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-18T22:21:35.888 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-18T22:21:35.888 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-18T22:21:35.888 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Preconfiguring packages ... 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 5,495 kB in 1s (5,352 kB/s) 2024-04-18T22:21:35.889 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package flex. 2024-04-18T22:21:35.918 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173891 files and directories currently installed.) 2024-04-18T22:21:35.925 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:35.970 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-18T22:21:35.990 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:36.526 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:36.658 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:36.661 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-04-18T22:21:36.691 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-18T22:21:36.718 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-18T22:21:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-18T22:21:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-18T22:21:37.094 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:37.106 INFO:teuthology.orchestra.run.smithi196.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-18T22:21:37.106 INFO:teuthology.orchestra.run.smithi196.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-18T22:21:37.106 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:21:37.106 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:21:37.106 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:21:37.106 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:21:37.107 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:21:37.107 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:21:37.107 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:21:37.108 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-18T22:21:37.108 INFO:teuthology.orchestra.run.smithi196.stdout: libfl-dev libfl2 libssl3 libuuid1 2024-04-18T22:21:37.110 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-18T22:21:37.110 INFO:teuthology.orchestra.run.smithi196.stdout: bison-doc flex-doc libssl-doc 2024-04-18T22:21:37.138 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-18T22:21:37.140 INFO:teuthology.orchestra.run.smithi196.stdout: bison flex libaio-dev libelf-dev libfl-dev libfl2 libssl-dev 2024-04-18T22:21:37.141 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-04-18T22:21:37.142 INFO:teuthology.orchestra.run.smithi196.stdout: libssl3 libuuid1 uuid-dev 2024-04-18T22:21:37.322 INFO:teuthology.orchestra.run.smithi196.stdout:3 upgraded, 7 newly installed, 0 to remove and 299 not upgraded. 2024-04-18T22:21:37.322 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 5,495 kB of archives. 2024-04-18T22:21:37.322 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 16.5 MB of additional disk space will be used. 2024-04-18T22:21:37.322 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-18T22:21:37.611 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:37.616 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:37.748 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:37.802 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-18T22:21:38.059 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-18T22:21:38.060 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-18T22:21:38.062 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-18T22:21:38.097 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-18T22:21:38.097 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-18T22:21:38.099 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-18T22:21:38.100 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-18T22:21:38.100 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-18T22:21:38.189 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:38.329 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:38.511 INFO:teuthology.orchestra.run.smithi196.stdout:Preconfiguring packages ... 2024-04-18T22:21:38.617 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 5,495 kB in 1s (5,374 kB/s) 2024-04-18T22:21:38.664 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:38.706 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package flex. 2024-04-18T22:21:38.769 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173891 files and directories currently installed.) 2024-04-18T22:21:38.774 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:38.814 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-18T22:21:39.031 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package bison. 2024-04-18T22:21:39.135 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:39.141 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../0-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-18T22:21:39.192 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:39.364 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:39.517 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-18T22:21:39.536 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-18T22:21:39.544 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../1-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-18T22:21:39.584 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:39.837 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-18T22:21:39.853 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../2-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-18T22:21:39.896 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:39.939 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:40.241 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-18T22:21:40.268 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../3-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:40.307 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:40.451 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:40.456 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-18T22:21:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../4-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:40.593 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:40.719 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:40.957 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-18T22:21:40.989 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-18T22:21:41.005 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../5-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:41.047 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:41.100 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-18T22:21:41.452 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:41.560 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-18T22:21:41.685 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:41.790 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package bison. 2024-04-18T22:21:41.820 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:41.863 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173964 files and directories currently installed.) 2024-04-18T22:21:41.868 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../0-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-18T22:21:41.910 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:41.946 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:42.064 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:42.173 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:42.236 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-18T22:21:42.263 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../1-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-18T22:21:42.290 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:42.294 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:42.380 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-18T22:21:42.450 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:42.547 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-18T22:21:42.563 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../2-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-18T22:21:42.577 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:42.622 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:42.728 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:42.909 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-18T22:21:42.937 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../3-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:42.967 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:43.220 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-18T22:21:43.247 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../4-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-18T22:21:43.286 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:43.538 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-18T22:21:43.566 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../5-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-18T22:21:43.605 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:43.946 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:44.100 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-18T22:21:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-18T22:21:44.352 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-18T22:21:44.478 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-18T22:21:44.488 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:44.493 DEBUG:teuthology.parallel:result is None 2024-04-18T22:21:44.579 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-18T22:21:44.680 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:44.789 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-18T22:21:44.878 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-18T22:21:44.943 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-18T22:21:45.070 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T22:21:45.221 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T22:21:46.389 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T22:21:46.915 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T22:21:46.920 DEBUG:teuthology.parallel:result is None 2024-04-18T22:21:46.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:21:47.058 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2024-04-18T22:21:47.079 INFO:teuthology.orchestra.run.smithi032.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.080 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.080 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T22:21:47.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:21:47.216 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-query -W -f '${Version}' ceph 2024-04-18T22:21:47.238 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.238 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.238 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T22:21:47.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:21:47.451 DEBUG:teuthology.orchestra.run.smithi096:> dpkg-query -W -f '${Version}' ceph 2024-04-18T22:21:47.470 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.470 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.470 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T22:21:47.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:21:47.608 DEBUG:teuthology.orchestra.run.smithi196:> dpkg-query -W -f '${Version}' ceph 2024-04-18T22:21:47.627 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.628 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T22:21:47.628 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T22:21:47.630 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-18T22:21:47.630 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:47.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T22:21:47.645 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:47.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T22:21:47.660 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:47.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T22:21:47.675 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:47.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T22:21:47.688 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-18T22:21:47.688 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:47.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T22:21:47.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T22:21:47.761 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:47.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T22:21:47.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T22:21:47.827 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:47.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T22:21:47.841 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T22:21:47.895 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:47.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T22:21:47.908 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T22:21:47.963 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-18T22:21:47.964 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:47.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T22:21:47.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T22:21:48.036 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:48.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T22:21:48.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T22:21:48.103 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:48.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T22:21:48.116 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T22:21:48.171 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:48.171 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T22:21:48.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T22:21:48.236 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-18T22:21:48.236 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:48.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T22:21:48.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T22:21:48.304 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:48.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T22:21:48.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T22:21:48.371 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:48.371 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T22:21:48.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T22:21:48.439 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:48.440 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T22:21:48.453 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T22:21:48.508 INFO:teuthology.run_tasks:Running task ceph... 2024-04-18T22:21:48.598 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-18T22:21:48.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2024-04-18T22:21:48.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-04-18T22:21:48.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-04-18T22:21:48.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-04-18T22:21:48.618 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-18T22:21:48.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T22:21:48.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T22:21:48.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T22:21:48.663 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T22:21:48.675 INFO:tasks.ceph:Creating extra log directories... 2024-04-18T22:21:48.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T22:21:48.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T22:21:48.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T22:21:48.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T22:21:48.744 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-18T22:21:48.744 INFO:tasks.ceph:config {'conf': {'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-18T22:21:48.744 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815', 'branch': 'squid', 'description': 'powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/ubuntu_latest tasks/admin_socket_objecter_requests thrashosds-health}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7662815', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd0bf63dd179c05751e4be49477389aa290c73089'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'], ['osd.0'], ['osd.1'], ['osd.2']], 'seed': 3123, 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32', 'sleep_before_teardown': 0, 'subset': '2/4', 'suite': 'powercycle', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd0bf63dd179c05751e4be49477389aa290c73089', 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbH8GKYerZQQDCKKdwmL/B1tET5gfWTb3XjPHYxUmrJk8gDxj5gUtY+wVDry20xQrTUsW1x9Uy5v8409hqsAB0=', 'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCvJJPlGl728mrthu3bgEjtyqIpgscw+UcRys7a+eQDk1cwxArsTUymySL7Io/44TMwx+MtsJzo+B9p3Td7zBGo=', 'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhdpWrsge4ltjd4TTC93pKKwl5G/UHxfWPnieTgbSYJPGMI3suRRBaQ0BE0cIhdir8CUR9GZxCXRXL528v9Epo=', 'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKv3SFUyG8tl2FjpHsPmjOvqm7BKPU1qsnd/gKJtDtiPNZKhyVtpZYEu5R5ykBHXxtVnbrV/+dNMFlPKQvvUXKU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}}, {'ceph': None}, {'thrashosds': {'chance_down': 1.0, 'powercycle': True, 'timeout': 600}}, {'radosbench': {'clients': ['client.0'], 'time': 60}}, {'admin_socket': {'client.0': {'objecter_requests': {'test': 'http://git.ceph.com/?p={repo};a=blob_plain;f=src/test/admin_socket/objecter_requests;hb={branch}'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-18_21:32:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-18T22:21:48.745 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T22:21:48.770 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T22:21:48.772 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T22:21:48.775 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T22:21:48.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T22:21:48.818 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T22:21:48.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T22:21:48.823 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T22:21:48.837 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:48.837 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T22:21:48.881 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T22:21:48.881 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-18 22:15:38.976155477 +0000 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-18 22:15:38.704157908 +0000 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-18 22:15:38.704157908 +0000 2024-04-18T22:21:48.929 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-04-18T22:21:48.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T22:21:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-18T22:21:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-18T22:21:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000417437 s, 1.2 MB/s 2024-04-18T22:21:48.984 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T22:21:49.034 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-18T22:21:49.080 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T22:21:49.080 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:49.080 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-18T22:21:49.080 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:49.080 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-18 22:15:39.456151190 +0000 2024-04-18T22:21:49.081 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-18 22:15:39.196153512 +0000 2024-04-18T22:21:49.081 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-18 22:15:39.196153512 +0000 2024-04-18T22:21:49.081 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-04-18T22:21:49.081 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T22:21:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-18T22:21:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-18T22:21:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000364733 s, 1.4 MB/s 2024-04-18T22:21:49.136 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T22:21:49.185 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-18T22:21:49.232 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T22:21:49.232 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:49.232 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-04-18T22:21:49.232 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:49.232 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-18 22:15:39.944146832 +0000 2024-04-18T22:21:49.232 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-18 22:15:39.700149011 +0000 2024-04-18T22:21:49.233 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-18 22:15:39.700149011 +0000 2024-04-18T22:21:49.233 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-04-18T22:21:49.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T22:21:49.286 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-18T22:21:49.286 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-18T22:21:49.286 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000299657 s, 1.7 MB/s 2024-04-18T22:21:49.287 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T22:21:49.337 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-18 22:15:40.200144545 +0000 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-18 22:15:40.196144581 +0000 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-18 22:15:40.196144581 +0000 2024-04-18T22:21:49.385 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2024-04-18T22:21:49.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T22:21:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-18T22:21:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-18T22:21:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000287343 s, 1.8 MB/s 2024-04-18T22:21:49.439 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T22:21:49.491 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-18T22:21:49.491 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:49.491 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T22:21:49.497 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T22:21:49.497 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-04-18T22:21:49.544 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T22:21:49.544 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:49.544 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-04-18T22:21:49.544 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:49.544 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-18 22:15:43.998376278 +0000 2024-04-18T22:21:49.544 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-18 22:15:43.738378597 +0000 2024-04-18T22:21:49.545 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-18 22:15:43.738378597 +0000 2024-04-18T22:21:49.545 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-18T22:21:49.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T22:21:49.598 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-18T22:21:49.599 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-18T22:21:49.599 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000448764 s, 1.1 MB/s 2024-04-18T22:21:49.600 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T22:21:49.649 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-18 22:15:44.498371817 +0000 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-18 22:15:44.246374066 +0000 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-18 22:15:44.246374066 +0000 2024-04-18T22:21:49.696 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-18T22:21:49.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T22:21:49.750 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-18T22:21:49.750 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-18T22:21:49.750 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000313704 s, 1.6 MB/s 2024-04-18T22:21:49.751 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T22:21:49.801 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-04-18T22:21:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T22:21:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:49.848 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-18T22:21:49.849 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:49.849 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-18 22:15:44.982367499 +0000 2024-04-18T22:21:49.849 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-18 22:15:44.722369820 +0000 2024-04-18T22:21:49.849 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-18 22:15:44.722369820 +0000 2024-04-18T22:21:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-18T22:21:49.849 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T22:21:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-18T22:21:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-18T22:21:49.902 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00036509 s, 1.4 MB/s 2024-04-18T22:21:49.904 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T22:21:49.954 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-04-18T22:21:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T22:21:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:50.000 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-04-18T22:21:50.000 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:50.000 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-18 22:15:45.502362861 +0000 2024-04-18T22:21:50.000 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-18 22:15:45.206365502 +0000 2024-04-18T22:21:50.001 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-18 22:15:45.206365502 +0000 2024-04-18T22:21:50.001 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-04-18T22:21:50.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T22:21:50.053 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-18T22:21:50.054 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-18T22:21:50.054 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000314556 s, 1.6 MB/s 2024-04-18T22:21:50.055 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T22:21:50.105 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1'} 2024-04-18T22:21:50.105 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:50.105 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T22:21:50.111 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T22:21:50.111 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-18T22:21:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-18 22:15:40.941823524 +0000 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-18 22:15:40.673825918 +0000 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-18 22:15:40.673825918 +0000 2024-04-18T22:21:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-18T22:21:50.158 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T22:21:50.212 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-18T22:21:50.212 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-18T22:21:50.212 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000406246 s, 1.3 MB/s 2024-04-18T22:21:50.213 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T22:21:50.262 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-18 22:15:41.453818950 +0000 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-18 22:15:41.189821308 +0000 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-18 22:15:41.189821308 +0000 2024-04-18T22:21:50.309 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-18T22:21:50.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T22:21:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-18T22:21:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-18T22:21:50.363 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000342026 s, 1.5 MB/s 2024-04-18T22:21:50.364 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T22:21:50.410 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-18 22:15:41.717816591 +0000 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-18 22:15:41.713816628 +0000 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-18 22:15:41.713816628 +0000 2024-04-18T22:21:50.457 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-18T22:21:50.458 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T22:21:50.512 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-18T22:21:50.512 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-18T22:21:50.512 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000426483 s, 1.2 MB/s 2024-04-18T22:21:50.513 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T22:21:50.563 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-18T22:21:50.609 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-18 22:15:42.493809661 +0000 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-18 22:15:42.245811876 +0000 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-18 22:15:42.245811876 +0000 2024-04-18T22:21:50.610 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-18T22:21:50.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T22:21:50.664 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-18T22:21:50.665 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-18T22:21:50.665 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000388567 s, 1.3 MB/s 2024-04-18T22:21:50.666 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T22:21:50.714 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1'} 2024-04-18T22:21:50.715 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1'}} 2024-04-18T22:21:50.715 INFO:tasks.ceph:Generating config... 2024-04-18T22:21:50.716 INFO:tasks.ceph:[client.0] admin socket = /var/run/ceph/ceph-$name.asok 2024-04-18T22:21:50.716 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-18T22:21:50.716 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-18T22:21:50.716 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-18T22:21:50.716 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-18T22:21:50.717 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-18T22:21:50.717 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-18T22:21:50.718 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-18T22:21:50.718 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-18T22:21:50.718 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-18T22:21:50.718 INFO:tasks.ceph:Setting up mon.a... 2024-04-18T22:21:50.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-18T22:21:50.754 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2024-04-18T22:21:50.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-18T22:21:50.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T22:21:50.908 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]'), ('mon.c', '[v2:172.21.15.32:3302,v1:172.21.15.32:6791]')] 2024-04-18T22:21:50.908 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}} 2024-04-18T22:21:50.909 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:50.909 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-18T22:21:50.956 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --addv b '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]' --addv c '[v2:172.21.15.32:3302,v1:172.21.15.32:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:51.042 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:fsid a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2024-04-18T22:21:51.045640+0000 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-18T22:21:51.045640+0000 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 17 (quincy) 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] mon.b 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] mon.c 2024-04-18T22:21:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-18T22:21:51.045 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-18T22:21:51.093 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a1b9e9bc-8f1f-4426-9796-fae392070eec... 2024-04-18T22:21:51.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T22:21:51.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T22:21:51.140 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T22:21:51.143 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-18T22:21:51.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T22:21:51.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-18T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]" 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-18T22:21:51.171 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-18T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-18T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression algorithm = zstd 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression mode = aggressive 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: osd blocked scrub grace period = 3600 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: osd max markdown count = 1000 2024-04-18T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-18T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T22:21:51.176 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-18T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-18T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-18T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-18T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-18T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/ceph-$name.asok 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-18T22:21:51.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-18T22:21:51.180 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.181 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]" 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-18T22:21:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-18T22:21:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore compression algorithm = zstd 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore compression mode = aggressive 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 1/20 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 1/20 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-18T22:21:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 4/10 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: osd blocked scrub grace period = 3600 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: osd max markdown count = 1000 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-18T22:21:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-18T22:21:51.188 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/ceph-$name.asok 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-04-18T22:21:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]" 2024-04-18T22:21:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-04-18T22:21:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug trim objects = true 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-04-18T22:21:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression algorithm = zstd 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression mode = aggressive 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 1/20 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 1/20 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 4/10 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T22:21:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: osd blocked scrub grace period = 3600 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: osd max markdown count = 1000 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-04-18T22:21:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T22:21:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-04-18T22:21:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-04-18T22:21:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/ceph-$name.asok 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.201 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.202 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-18T22:21:51.203 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]" 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-18T22:21:51.204 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.205 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-18T22:21:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore compression algorithm = zstd 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore compression mode = aggressive 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: osd blocked scrub grace period = 3600 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-18T22:21:51.207 INFO:teuthology.orchestra.run.smithi086.stdout: osd max markdown count = 1000 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-18T22:21:51.208 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.209 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T22:21:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T22:21:51.211 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-18T22:21:51.211 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-18T22:21:51.211 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-18T22:21:51.211 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-18T22:21:51.211 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-18T22:21:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/ceph-$name.asok 2024-04-18T22:21:51.213 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-18T22:21:51.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-18T22:21:51.307 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-18T22:21:51.307 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:51.308 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-18T22:21:51.313 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:51.313 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-18T22:21:51.360 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2024-04-18T22:21:51.360 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:51.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T22:21:51.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T22:21:51.427 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:21:51.427 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:51.473 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-04-18T22:21:51.473 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:51.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T22:21:51.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T22:21:51.497 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:21:51.497 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:51.545 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:21:51.545 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:51.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T22:21:51.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T22:21:51.568 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:21:51.568 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:51.616 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-04-18T22:21:51.616 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:51.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T22:21:51.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T22:21:51.640 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:21:51.640 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:51.689 INFO:tasks.ceph:Setting up mon nodes... 2024-04-18T22:21:51.689 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-18T22:21:51.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-18T22:21:51.735 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-18T22:21:51.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-18T22:21:51.821 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-18T22:21:51.824 INFO:tasks.ceph:Setting up mds nodes... 2024-04-18T22:21:51.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-18T22:21:51.907 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-18T22:21:51.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-18T22:21:51.964 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-18T22:21:51.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-18T22:21:52.042 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-18T22:21:52.053 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-18T22:21:52.053 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1'}} 2024-04-18T22:21:52.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-18T22:21:52.068 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-18T22:21:52.068 INFO:tasks.ceph:role: osd.0 2024-04-18T22:21:52.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-18T22:21:52.068 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T22:21:52.133 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T22:21:52.591 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-18T22:21:52.593 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-18T22:21:52.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-18T22:21:52.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-18T22:21:52.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:21:52.700 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T22:21:52.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-18T22:21:52.806+0000 7ff4b390e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-18T22:21:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-18T22:21:52.806+0000 7ff4b390e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-18T22:21:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-18T22:21:52.806+0000 7ff4b390e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-18T22:21:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-18T22:21:52.806+0000 7ff4b390e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-18T22:21:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-18T22:21:52.806+0000 7ff4b390e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-18T22:21:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-18T22:21:52.814+0000 7ff4b390e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-18T22:21:55.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-18T22:21:55.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-18T22:21:55.830 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1'} 2024-04-18T22:21:55.831 INFO:tasks.ceph:role: osd.1 2024-04-18T22:21:55.831 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:21:55.831 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T22:21:55.893 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T22:21:56.349 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-04-18T22:21:56.351 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-04-18T22:21:56.351 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-18T22:21:56.439 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-18T22:21:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T22:21:56.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:21:56.494 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:21:56.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-18T22:21:56.599+0000 7f41891bd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-18T22:21:56.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-18T22:21:56.603+0000 7f41891bd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-18T22:21:56.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-18T22:21:56.603+0000 7f41891bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-18T22:21:56.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-18T22:21:56.603+0000 7f41891bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-18T22:21:56.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-18T22:21:56.603+0000 7f41891bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-18T22:21:56.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-18T22:21:56.607+0000 7f41891bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-18T22:21:59.611 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-18T22:21:59.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-18T22:21:59.639 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1'} 2024-04-18T22:21:59.639 INFO:tasks.ceph:role: osd.2 2024-04-18T22:21:59.639 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-18T22:21:59.639 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T22:21:59.700 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T22:21:59.701 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T22:22:00.158 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-18T22:22:00.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-18T22:22:00.161 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-18T22:22:00.257 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-18T22:22:00.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:22:00.268 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T22:22:00.268 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:22:00.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-18T22:22:00.371+0000 7f23cd02d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-18T22:22:00.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-18T22:22:00.375+0000 7f23cd02d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-18T22:22:00.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-18T22:22:00.375+0000 7f23cd02d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-18T22:22:00.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-18T22:22:00.375+0000 7f23cd02d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-18T22:22:00.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-18T22:22:00.375+0000 7f23cd02d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-18T22:22:00.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-18T22:22:00.379+0000 7f23cd02d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-18T22:22:03.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-18T22:22:03.385 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-18T22:22:03.385 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:22:03.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-18T22:22:03.400 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:22:03.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-18T22:22:03.457 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:22:03.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-18T22:22:03.512 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:22:03.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-18T22:22:03.526 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:22:03.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-18T22:22:03.540 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:22:03.540 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-18T22:22:03.554 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-18T22:22:03.554 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-18T22:22:03.560 INFO:tasks.ceph:Adding keys to all mons... 2024-04-18T22:22:03.561 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD/nCFmkLIJLBAAGAZJOk3q4lgcX8Z7X4scUA== 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD/nCFmfH8nMRAALRHQmmJ2WNIKWD6C8Eg7Iw== 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQD/nCFmbwZFNhAAm+AQZc0nI+bWuQwqPFiP7g== 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAAnSFmeKctMBAAnUKQhypTFi595oKTtLGa5A== 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAEnSFmQervIxAAWqrsU5+9MOisnfUxWvsRvA== 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2024-04-18T22:22:03.615 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAInSFmYV5hFhAAuBFsiTEwtk1TsgXCZDmQxw== 2024-04-18T22:22:03.616 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-18T22:22:03.616 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAAnSFmvqa3AhAALl1+oGoTevPnIYRJjkjwqw== 2024-04-18T22:22:03.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T22:22:03.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T22:22:03.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-18T22:22:03.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T22:22:03.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T22:22:03.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T22:22:04.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-18T22:22:04.146 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-18T22:22:04.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-18T22:22:04.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-18T22:22:04.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-18T22:22:04.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-18T22:22:04.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-18T22:22:04.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-18T22:22:04.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-18T22:22:04.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-18T22:22:04.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-18T22:22:04.587 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-18T22:22:04.635 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-18T22:22:04.636 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-18T22:22:04.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-18T22:22:04.681 INFO:tasks.ceph.mon.a:Started 2024-04-18T22:22:04.682 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-18T22:22:04.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-18T22:22:04.684 INFO:tasks.ceph.mon.b:Started 2024-04-18T22:22:04.684 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-18T22:22:04.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-18T22:22:04.686 INFO:tasks.ceph.mon.c:Started 2024-04-18T22:22:04.686 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-18T22:22:04.686 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-18T22:22:04.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-18T22:22:04.688 INFO:tasks.ceph.mgr.x:Started 2024-04-18T22:22:04.688 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-18T22:22:04.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-18T22:22:04.691 INFO:tasks.ceph.mgr.y:Started 2024-04-18T22:22:04.691 INFO:tasks.ceph:Setting crush tunables to default 2024-04-18T22:22:04.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2024-04-18T22:22:10.273 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:10.273+0000 7f229b5fb180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-18T22:22:10.754 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:10.753+0000 7f229b5fb180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-18T22:22:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-04-18T22:22:10.933 INFO:tasks.ceph:check_enable_crimson: False 2024-04-18T22:22:10.933 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-18T22:22:10.933 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-18T22:22:10.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-18T22:22:10.947 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-18T22:22:10.948 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-18T22:22:10.961 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-18T22:22:10.961 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-18T22:22:10.969 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:10.969+0000 7f229b5fb180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-18T22:22:10.975 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 57238913-8d4f-4ea8-a1a5-92ce7aceaf4a 0 2024-04-18T22:22:11.050 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:11.049+0000 7f08b72e6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-18T22:22:11.231 INFO:teuthology.orchestra.run.smithi196.stdout:0 2024-04-18T22:22:11.250 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 68bfe465-447e-4083-a8d5-e6f996b2bb2c 1 2024-04-18T22:22:11.447 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:11.445+0000 7f08b72e6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-18T22:22:11.454 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-18T22:22:11.472 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e 2 2024-04-18T22:22:11.566 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.565+0000 7f229b5fb180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-18T22:22:11.592 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:11.593+0000 7f08b72e6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-18T22:22:11.626 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.625+0000 7f229b5fb180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-18T22:22:11.683 INFO:teuthology.orchestra.run.smithi196.stdout:2 2024-04-18T22:22:11.683 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.681+0000 7f229b5fb180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-18T22:22:11.701 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-18T22:22:11.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-18T22:22:11.703 INFO:tasks.ceph.osd.0:Started 2024-04-18T22:22:11.703 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-18T22:22:11.703 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-18T22:22:11.706 INFO:tasks.ceph.osd.1:Started 2024-04-18T22:22:11.706 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-18T22:22:11.706 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-18T22:22:11.708 INFO:tasks.ceph.osd.2:Started 2024-04-18T22:22:11.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:11.766 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.765+0000 7f229b5fb180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-18T22:22:11.824 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.825+0000 7f229b5fb180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-18T22:22:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:11.907 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:11.681436+0000","last_up_change":"0.000000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:11.910 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.909+0000 7f229b5fb180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-18T22:22:11.923 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-18T22:22:11.923 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-18T22:22:11.967 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:11.965+0000 7f229b5fb180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-18T22:22:12.044 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.045+0000 7f08b72e6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-18T22:22:12.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.097+0000 7f08b72e6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-18T22:22:12.128 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:12.129+0000 7f229b5fb180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-18T22:22:12.151 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.149+0000 7f08b72e6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-18T22:22:12.193 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:12.193+0000 7f229b5fb180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-18T22:22:12.225 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:12.230 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.229+0000 7f08b72e6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-18T22:22:12.283 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.281+0000 7f08b72e6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-18T22:22:12.321 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:12.322+0000 7fa29c23c7c0 -1 Falling back to public interface 2024-04-18T22:22:12.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:12.323+0000 7f57ee07b7c0 -1 Falling back to public interface 2024-04-18T22:22:12.324 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:12.323+0000 7fe88e2947c0 -1 Falling back to public interface 2024-04-18T22:22:12.362 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.361+0000 7f08b72e6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-18T22:22:12.407 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-18T22:22:12.407 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":5,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:11.681436+0000","last_up_change":"0.000000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:12.415 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.413+0000 7f08b72e6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-18T22:22:12.422 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-18T22:22:12.619 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.617+0000 7f08b72e6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-18T22:22:12.674 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.673+0000 7f08b72e6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-18T22:22:12.883 INFO:tasks.ceph.mgr.x.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-18T22:22:12.883 INFO:tasks.ceph.mgr.x.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-18T22:22:12.883 INFO:tasks.ceph.mgr.x.smithi032.stderr: from numpy import show_config as show_numpy_config 2024-04-18T22:22:12.904 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-18T22:22:12.905 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-18T22:22:12.905 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2024-04-18T22:22:12.925 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:12.925+0000 7f229b5fb180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-18T22:22:12.925 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:12.925+0000 7f08b72e6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-18T22:22:13.066 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:13.065+0000 7f229b5fb180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-18T22:22:13.067 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:13.065+0000 7f08b72e6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-18T22:22:13.242 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:13.241+0000 7f229b5fb180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-18T22:22:13.242 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:13.241+0000 7f08b72e6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-18T22:22:13.309 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:13.309+0000 7f229b5fb180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-18T22:22:13.310 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:13.309+0000 7f08b72e6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-18T22:22:13.451 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:13.449+0000 7f08b72e6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-18T22:22:13.451 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:13.449+0000 7f229b5fb180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-18T22:22:14.089 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:14.089+0000 7f08b72e6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-18T22:22:14.089 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:14.089+0000 7f229b5fb180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-18T22:22:14.169 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:14.169+0000 7f08b72e6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-18T22:22:14.169 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:14.169+0000 7f229b5fb180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-18T22:22:14.361 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:14.361+0000 7f229b5fb180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-18T22:22:14.361 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:14.361+0000 7f08b72e6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-18T22:22:14.439 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:14.437+0000 7f229b5fb180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-18T22:22:14.439 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:14.437+0000 7f08b72e6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-18T22:22:14.745 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:14.745+0000 7f08b72e6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-18T22:22:14.745 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:14.745+0000 7f229b5fb180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-18T22:22:14.810 INFO:tasks.ceph.mgr.x.smithi032.stderr:2024-04-18T22:22:14.809+0000 7f229b5fb180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-18T22:22:14.811 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:14.809+0000 7f08b72e6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-18T22:22:15.844 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:15.845+0000 7f085e731640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-18T22:22:15.844 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:22:15.845+0000 7f085e731640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-18T22:22:15.975 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:15.974+0000 7fa29c23c7c0 -1 osd.0 0 log_to_monitors true 2024-04-18T22:22:16.008 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:16.007+0000 7f57ee07b7c0 -1 osd.2 0 log_to_monitors true 2024-04-18T22:22:16.029 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:16.027+0000 7fe88e2947c0 -1 osd.1 0 log_to_monitors true 2024-04-18T22:22:18.650 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:18.650+0000 7fa2989f7640 -1 osd.0 0 waiting for initial osdmap 2024-04-18T22:22:18.677 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:18.678+0000 7fa29001a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T22:22:18.682 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:18.683+0000 7f57ea024640 -1 osd.2 0 waiting for initial osdmap 2024-04-18T22:22:18.698 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:18.695+0000 7fe88a23d640 -1 osd.1 0 waiting for initial osdmap 2024-04-18T22:22:18.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:18.707+0000 7f57e1e59640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T22:22:18.720 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:18.719+0000 7fe882072640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T22:22:18.724 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:19.028 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-18T22:22:19.028 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":7,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:17.883322+0000","last_up_change":"0.000000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:19.046 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-18T22:22:25.348 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:25.649 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-18T22:22:25.649 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":11,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:22.922790+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:25.666 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-04-18T22:22:25.667 INFO:tasks.ceph:Creating RBD pool 2024-04-18T22:22:25.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-18T22:22:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'rbd' created 2024-04-18T22:22:26.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-18T22:22:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-18T22:22:27.099 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-18T22:22:27.100 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-18T22:22:27.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-18T22:22:27.102 INFO:tasks.ceph.mds.a:Started 2024-04-18T22:22:27.102 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-18T22:22:27.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:27.182 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-18T22:22:27.396 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:27.396 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:27.069188+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-18T22:22:20.901399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-18T22:22:26.016101+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-18T22:22:27.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-18T22:22:27.703 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-18T22:22:27.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-18T22:22:28.013 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2024-04-18T22:22:28.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:22:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:28.363 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-18T22:22:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2024-04-18T22:22:28.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:28.672 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":14,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:28.077097+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:28.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-18T22:22:20.901399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-18T22:22:26.016101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-18T22:22:28.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-18T22:22:28.984 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-18T22:22:29.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-18T22:22:29.300 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2024-04-18T22:22:29.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:29.614 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2024-04-18T22:22:29.615 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2024-04-18T22:22:29.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:22:29.615 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-18T22:22:29.630 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-18T22:22:29.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-18T22:22:30.094 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-18T22:22:30.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-18T22:22:31.109 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-18T22:22:31.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-18T22:22:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 3 and data pool 4 2024-04-18T22:22:32.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:31.467246+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:32.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:32.739 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:31.467246+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:32.758 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-18T22:22:33.144 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-18T22:22:33.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-18T22:22:34.149 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-18T22:22:34.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:22:34.482 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:34.483 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-18T22:22:31.467948+0000","modified":"2024-04-18T22:22:33.473846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-04-18T22:22:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-18T22:22:34.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:34.785 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:34.785 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:33.115398+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:34.802 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-18T22:22:34.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:35.100 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:35.102 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:35.116 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-18T22:22:34.828856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85712,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029808,"statfs":{"total":289910292480,"available":289822523392,"internally_reserved":0,"allocated":483328,"data_stored":1291175,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":87223631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.734429"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":214,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118395+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:28.118395+0000","last_peered":"2024-04-18T22:22:28.118395+0000","last_clean":"2024-04-18T22:22:28.118395+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:28.118395+0000","last_undegraded":"2024-04-18T22:22:28.118395+0000","last_fullsized":"2024-04-18T22:22:28.118395+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118533+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:28.118533+0000","last_peered":"2024-04-18T22:22:28.118533+0000","last_clean":"2024-04-18T22:22:28.118533+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:28.118533+0000","last_undegraded":"2024-04-18T22:22:28.118533+0000","last_fullsized":"2024-04-18T22:22:28.118533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118669+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:28.118669+0000","last_peered":"2024-04-18T22:22:28.118669+0000","last_clean":"2024-04-18T22:22:28.118669+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:28.118669+0000","last_undegraded":"2024-04-18T22:22:28.118669+0000","last_fullsized":"2024-04-18T22:22:28.118669+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082233+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:28.082233+0000","last_peered":"2024-04-18T22:22:28.082233+0000","last_clean":"2024-04-18T22:22:28.082233+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:28.082233+0000","last_undegraded":"2024-04-18T22:22:28.082233+0000","last_fullsized":"2024-04-18T22:22:28.082233+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118692+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:28.118692+0000","last_peered":"2024-04-18T22:22:28.118692+0000","last_clean":"2024-04-18T22:22:28.118692+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:28.118692+0000","last_undegraded":"2024-04-18T22:22:28.118692+0000","last_fullsized":"2024-04-18T22:22:28.118692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082306+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:28.082306+0000","last_peered":"2024-04-18T22:22:28.082306+0000","last_clean":"2024-04-18T22:22:28.082306+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:28.082306+0000","last_undegraded":"2024-04-18T22:22:28.082306+0000","last_fullsized":"2024-04-18T22:22:28.082306+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.784991+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:28.784991+0000","last_peered":"2024-04-18T22:22:28.784991+0000","last_clean":"2024-04-18T22:22:28.784991+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:28.784991+0000","last_undegraded":"2024-04-18T22:22:28.784991+0000","last_fullsized":"2024-04-18T22:22:28.784991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082305+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:28.082305+0000","last_peered":"2024-04-18T22:22:28.082305+0000","last_clean":"2024-04-18T22:22:28.082305+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:28.082305+0000","last_undegraded":"2024-04-18T22:22:28.082305+0000","last_fullsized":"2024-04-18T22:22:28.082305+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118511+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:28.118511+0000","last_peered":"2024-04-18T22:22:28.118511+0000","last_clean":"2024-04-18T22:22:28.118511+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:28.118511+0000","last_undegraded":"2024-04-18T22:22:28.118511+0000","last_fullsized":"2024-04-18T22:22:28.118511+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":122880,"data_stored":36813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:35.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:35.448 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:35.465 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-18T22:22:34.828856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85712,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029808,"statfs":{"total":289910292480,"available":289822523392,"internally_reserved":0,"allocated":483328,"data_stored":1291175,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":87223631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.734429"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":214,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118395+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:28.118395+0000","last_peered":"2024-04-18T22:22:28.118395+0000","last_clean":"2024-04-18T22:22:28.118395+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:28.118395+0000","last_undegraded":"2024-04-18T22:22:28.118395+0000","last_fullsized":"2024-04-18T22:22:28.118395+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118533+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:28.118533+0000","last_peered":"2024-04-18T22:22:28.118533+0000","last_clean":"2024-04-18T22:22:28.118533+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:28.118533+0000","last_undegraded":"2024-04-18T22:22:28.118533+0000","last_fullsized":"2024-04-18T22:22:28.118533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118669+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:28.118669+0000","last_peered":"2024-04-18T22:22:28.118669+0000","last_clean":"2024-04-18T22:22:28.118669+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:28.118669+0000","last_undegraded":"2024-04-18T22:22:28.118669+0000","last_fullsized":"2024-04-18T22:22:28.118669+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082233+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:28.082233+0000","last_peered":"2024-04-18T22:22:28.082233+0000","last_clean":"2024-04-18T22:22:28.082233+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:28.082233+0000","last_undegraded":"2024-04-18T22:22:28.082233+0000","last_fullsized":"2024-04-18T22:22:28.082233+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118692+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:28.118692+0000","last_peered":"2024-04-18T22:22:28.118692+0000","last_clean":"2024-04-18T22:22:28.118692+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:28.118692+0000","last_undegraded":"2024-04-18T22:22:28.118692+0000","last_fullsized":"2024-04-18T22:22:28.118692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082306+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:28.082306+0000","last_peered":"2024-04-18T22:22:28.082306+0000","last_clean":"2024-04-18T22:22:28.082306+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:28.082306+0000","last_undegraded":"2024-04-18T22:22:28.082306+0000","last_fullsized":"2024-04-18T22:22:28.082306+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.784991+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:28.784991+0000","last_peered":"2024-04-18T22:22:28.784991+0000","last_clean":"2024-04-18T22:22:28.784991+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:28.784991+0000","last_undegraded":"2024-04-18T22:22:28.784991+0000","last_fullsized":"2024-04-18T22:22:28.784991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082305+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:28.082305+0000","last_peered":"2024-04-18T22:22:28.082305+0000","last_clean":"2024-04-18T22:22:28.082305+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:28.082305+0000","last_undegraded":"2024-04-18T22:22:28.082305+0000","last_fullsized":"2024-04-18T22:22:28.082305+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118511+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:28.118511+0000","last_peered":"2024-04-18T22:22:28.118511+0000","last_clean":"2024-04-18T22:22:28.118511+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:28.118511+0000","last_undegraded":"2024-04-18T22:22:28.118511+0000","last_fullsized":"2024-04-18T22:22:28.118511+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":122880,"data_stored":36813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:35.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:35.806 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:35.807 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:35.824 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-18T22:22:34.828856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85712,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029808,"statfs":{"total":289910292480,"available":289822523392,"internally_reserved":0,"allocated":483328,"data_stored":1291175,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":87223631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.734429"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":214,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118395+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:28.118395+0000","last_peered":"2024-04-18T22:22:28.118395+0000","last_clean":"2024-04-18T22:22:28.118395+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:28.118395+0000","last_undegraded":"2024-04-18T22:22:28.118395+0000","last_fullsized":"2024-04-18T22:22:28.118395+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118533+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:28.118533+0000","last_peered":"2024-04-18T22:22:28.118533+0000","last_clean":"2024-04-18T22:22:28.118533+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:28.118533+0000","last_undegraded":"2024-04-18T22:22:28.118533+0000","last_fullsized":"2024-04-18T22:22:28.118533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118669+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:28.118669+0000","last_peered":"2024-04-18T22:22:28.118669+0000","last_clean":"2024-04-18T22:22:28.118669+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:28.118669+0000","last_undegraded":"2024-04-18T22:22:28.118669+0000","last_fullsized":"2024-04-18T22:22:28.118669+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082233+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:28.082233+0000","last_peered":"2024-04-18T22:22:28.082233+0000","last_clean":"2024-04-18T22:22:28.082233+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:28.082233+0000","last_undegraded":"2024-04-18T22:22:28.082233+0000","last_fullsized":"2024-04-18T22:22:28.082233+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118692+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:28.118692+0000","last_peered":"2024-04-18T22:22:28.118692+0000","last_clean":"2024-04-18T22:22:28.118692+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:28.118692+0000","last_undegraded":"2024-04-18T22:22:28.118692+0000","last_fullsized":"2024-04-18T22:22:28.118692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082306+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:28.082306+0000","last_peered":"2024-04-18T22:22:28.082306+0000","last_clean":"2024-04-18T22:22:28.082306+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:28.082306+0000","last_undegraded":"2024-04-18T22:22:28.082306+0000","last_fullsized":"2024-04-18T22:22:28.082306+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.784991+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:28.784991+0000","last_peered":"2024-04-18T22:22:28.784991+0000","last_clean":"2024-04-18T22:22:28.784991+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:28.784991+0000","last_undegraded":"2024-04-18T22:22:28.784991+0000","last_fullsized":"2024-04-18T22:22:28.784991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082305+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:28.082305+0000","last_peered":"2024-04-18T22:22:28.082305+0000","last_clean":"2024-04-18T22:22:28.082305+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:28.082305+0000","last_undegraded":"2024-04-18T22:22:28.082305+0000","last_fullsized":"2024-04-18T22:22:28.082305+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118511+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:28.118511+0000","last_peered":"2024-04-18T22:22:28.118511+0000","last_clean":"2024-04-18T22:22:28.118511+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:28.118511+0000","last_undegraded":"2024-04-18T22:22:28.118511+0000","last_fullsized":"2024-04-18T22:22:28.118511+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":122880,"data_stored":36813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:35.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3d is not active+clean 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3c is not active+clean 2024-04-18T22:22:35.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3a is not active+clean 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.39 is not active+clean 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.38 is not active+clean 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2024-04-18T22:22:35.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.36 is not active+clean 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.35 is not active+clean 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.32 is not active+clean 2024-04-18T22:22:35.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.31 is not active+clean 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.30 is not active+clean 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2e is not active+clean 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2024-04-18T22:22:35.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2c is not active+clean 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2b is not active+clean 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.28 is not active+clean 2024-04-18T22:22:35.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.25 is not active+clean 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2024-04-18T22:22:35.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.22 is not active+clean 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.20 is not active+clean 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-04-18T22:22:35.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2024-04-18T22:22:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-04-18T22:22:35.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2024-04-18T22:22:35.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-04-18T22:22:35.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-04-18T22:22:35.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2024-04-18T22:22:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2024-04-18T22:22:35.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:31.093710+0000', 'last_change': '2024-04-18T22:22:31.093710+0000', 'last_active': '2024-04-18T22:22:31.093710+0000', 'last_peered': '2024-04-18T22:22:31.093710+0000', 'last_clean': '2024-04-18T22:22:31.093710+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:31.093710+0000', 'last_undegraded': '2024-04-18T22:22:31.093710+0000', 'last_fullsized': '2024-04-18T22:22:31.093710+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:31.093710+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-18T22:22:35.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-18T22:22:30.083100+0000', 'last_change': '2024-04-18T22:22:30.083100+0000', 'last_active': '2024-04-18T22:22:30.083100+0000', 'last_peered': '2024-04-18T22:22:30.083100+0000', 'last_clean': '2024-04-18T22:22:30.083100+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-18T22:22:30.083100+0000', 'last_undegraded': '2024-04-18T22:22:30.083100+0000', 'last_fullsized': '2024-04-18T22:22:30.083100+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'last_clean_scrub_stamp': '2024-04-18T22:22:30.083100+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-18T22:22:35.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-18T22:22:36.228 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:36.228 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":26,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":18,"num_osds":3,"num_up_osds":3,"osd_up_since":1713478939,"num_in_osds":3,"osd_in_since":1713478931,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":87769088,"bytes_avail":289822523392,"bytes_total":289910292480,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":5,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4205}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-18T22:22:10.047201+0000","services":{}},"progress_events":{}} 2024-04-18T22:22:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-18T22:22:36.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:36.556 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:36.574 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-18T22:22:34.828856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85712,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029808,"statfs":{"total":289910292480,"available":289822523392,"internally_reserved":0,"allocated":483328,"data_stored":1291175,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":87223631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.734429"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":214,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118395+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:28.118395+0000","last_peered":"2024-04-18T22:22:28.118395+0000","last_clean":"2024-04-18T22:22:28.118395+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:28.118395+0000","last_undegraded":"2024-04-18T22:22:28.118395+0000","last_fullsized":"2024-04-18T22:22:28.118395+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118533+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:28.118533+0000","last_peered":"2024-04-18T22:22:28.118533+0000","last_clean":"2024-04-18T22:22:28.118533+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:28.118533+0000","last_undegraded":"2024-04-18T22:22:28.118533+0000","last_fullsized":"2024-04-18T22:22:28.118533+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118669+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:28.118669+0000","last_peered":"2024-04-18T22:22:28.118669+0000","last_clean":"2024-04-18T22:22:28.118669+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:28.118669+0000","last_undegraded":"2024-04-18T22:22:28.118669+0000","last_fullsized":"2024-04-18T22:22:28.118669+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082233+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:28.082233+0000","last_peered":"2024-04-18T22:22:28.082233+0000","last_clean":"2024-04-18T22:22:28.082233+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:28.082233+0000","last_undegraded":"2024-04-18T22:22:28.082233+0000","last_fullsized":"2024-04-18T22:22:28.082233+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118692+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:28.118692+0000","last_peered":"2024-04-18T22:22:28.118692+0000","last_clean":"2024-04-18T22:22:28.118692+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:28.118692+0000","last_undegraded":"2024-04-18T22:22:28.118692+0000","last_fullsized":"2024-04-18T22:22:28.118692+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082306+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:28.082306+0000","last_peered":"2024-04-18T22:22:28.082306+0000","last_clean":"2024-04-18T22:22:28.082306+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:28.082306+0000","last_undegraded":"2024-04-18T22:22:28.082306+0000","last_fullsized":"2024-04-18T22:22:28.082306+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.784991+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:28.784991+0000","last_peered":"2024-04-18T22:22:28.784991+0000","last_clean":"2024-04-18T22:22:28.784991+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:28.784991+0000","last_undegraded":"2024-04-18T22:22:28.784991+0000","last_fullsized":"2024-04-18T22:22:28.784991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.082305+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:28.082305+0000","last_peered":"2024-04-18T22:22:28.082305+0000","last_clean":"2024-04-18T22:22:28.082305+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:28.082305+0000","last_undegraded":"2024-04-18T22:22:28.082305+0000","last_fullsized":"2024-04-18T22:22:28.082305+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-18T22:22:28.118511+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:28.118511+0000","last_peered":"2024-04-18T22:22:28.118511+0000","last_clean":"2024-04-18T22:22:28.118511+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:28.118511+0000","last_undegraded":"2024-04-18T22:22:28.118511+0000","last_fullsized":"2024-04-18T22:22:28.118511+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:31.093710+0000","last_change":"2024-04-18T22:22:31.093710+0000","last_active":"2024-04-18T22:22:31.093710+0000","last_peered":"2024-04-18T22:22:31.093710+0000","last_clean":"2024-04-18T22:22:31.093710+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:31.093710+0000","last_undegraded":"2024-04-18T22:22:31.093710+0000","last_fullsized":"2024-04-18T22:22:31.093710+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-18T22:22:30.083100+0000","last_change":"2024-04-18T22:22:30.083100+0000","last_active":"2024-04-18T22:22:30.083100+0000","last_peered":"2024-04-18T22:22:30.083100+0000","last_clean":"2024-04-18T22:22:30.083100+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-18T22:22:30.083100+0000","last_undegraded":"2024-04-18T22:22:30.083100+0000","last_fullsized":"2024-04-18T22:22:30.083100+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":627181,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":122880,"data_stored":36813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:37.212 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-18T22:22:37.211+0000 7fc18edc0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-18T22:22:39.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:39.896 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:39.898 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:39.912 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-18T22:22:38.830128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":90020,"kb_used_data":632,"kb_used_omap":4,"kb_used_meta":89339,"kb_avail":283025500,"statfs":{"total":289910292480,"available":289818112000,"internally_reserved":0,"allocated":647168,"data_stored":1350892,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":91483471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.722541"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128432+0000","last_change":"2024-04-18T22:22:32.145997+0000","last_active":"2024-04-18T22:22:33.128432+0000","last_peered":"2024-04-18T22:22:33.128432+0000","last_clean":"2024-04-18T22:22:33.128432+0000","last_became_active":"2024-04-18T22:22:32.143852+0000","last_became_peered":"2024-04-18T22:22:32.143852+0000","last_unstale":"2024-04-18T22:22:33.128432+0000","last_undegraded":"2024-04-18T22:22:33.128432+0000","last_fullsized":"2024-04-18T22:22:33.128432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126921+0000","last_change":"2024-04-18T22:22:32.148788+0000","last_active":"2024-04-18T22:22:33.126921+0000","last_peered":"2024-04-18T22:22:33.126921+0000","last_clean":"2024-04-18T22:22:33.126921+0000","last_became_active":"2024-04-18T22:22:32.146781+0000","last_became_peered":"2024-04-18T22:22:32.146781+0000","last_unstale":"2024-04-18T22:22:33.126921+0000","last_undegraded":"2024-04-18T22:22:33.126921+0000","last_fullsized":"2024-04-18T22:22:33.126921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241643+0000","last_change":"2024-04-18T22:22:32.152874+0000","last_active":"2024-04-18T22:22:33.241643+0000","last_peered":"2024-04-18T22:22:33.241643+0000","last_clean":"2024-04-18T22:22:33.241643+0000","last_became_active":"2024-04-18T22:22:32.152154+0000","last_became_peered":"2024-04-18T22:22:32.152154+0000","last_unstale":"2024-04-18T22:22:33.241643+0000","last_undegraded":"2024-04-18T22:22:33.241643+0000","last_fullsized":"2024-04-18T22:22:33.241643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242573+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.242573+0000","last_peered":"2024-04-18T22:22:33.242573+0000","last_clean":"2024-04-18T22:22:33.242573+0000","last_became_active":"2024-04-18T22:22:32.142591+0000","last_became_peered":"2024-04-18T22:22:32.142591+0000","last_unstale":"2024-04-18T22:22:33.242573+0000","last_undegraded":"2024-04-18T22:22:33.242573+0000","last_fullsized":"2024-04-18T22:22:33.242573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:24:32.079309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127358+0000","last_change":"2024-04-18T22:22:32.136433+0000","last_active":"2024-04-18T22:22:33.127358+0000","last_peered":"2024-04-18T22:22:33.127358+0000","last_clean":"2024-04-18T22:22:33.127358+0000","last_became_active":"2024-04-18T22:22:32.135586+0000","last_became_peered":"2024-04-18T22:22:32.135586+0000","last_unstale":"2024-04-18T22:22:33.127358+0000","last_undegraded":"2024-04-18T22:22:33.127358+0000","last_fullsized":"2024-04-18T22:22:33.127358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240606+0000","last_change":"2024-04-18T22:22:32.134957+0000","last_active":"2024-04-18T22:22:33.240606+0000","last_peered":"2024-04-18T22:22:33.240606+0000","last_clean":"2024-04-18T22:22:33.240606+0000","last_became_active":"2024-04-18T22:22:32.133797+0000","last_became_peered":"2024-04-18T22:22:32.133797+0000","last_unstale":"2024-04-18T22:22:33.240606+0000","last_undegraded":"2024-04-18T22:22:33.240606+0000","last_fullsized":"2024-04-18T22:22:33.240606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240547+0000","last_change":"2024-04-18T22:22:32.133775+0000","last_active":"2024-04-18T22:22:33.240547+0000","last_peered":"2024-04-18T22:22:33.240547+0000","last_clean":"2024-04-18T22:22:33.240547+0000","last_became_active":"2024-04-18T22:22:32.132972+0000","last_became_peered":"2024-04-18T22:22:32.132972+0000","last_unstale":"2024-04-18T22:22:33.240547+0000","last_undegraded":"2024-04-18T22:22:33.240547+0000","last_fullsized":"2024-04-18T22:22:33.240547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.239986+0000","last_change":"2024-04-18T22:22:32.136411+0000","last_active":"2024-04-18T22:22:33.239986+0000","last_peered":"2024-04-18T22:22:33.239986+0000","last_clean":"2024-04-18T22:22:33.239986+0000","last_became_active":"2024-04-18T22:22:32.132809+0000","last_became_peered":"2024-04-18T22:22:32.132809+0000","last_unstale":"2024-04-18T22:22:33.239986+0000","last_undegraded":"2024-04-18T22:22:33.239986+0000","last_fullsized":"2024-04-18T22:22:33.239986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242905+0000","last_change":"2024-04-18T22:22:32.150575+0000","last_active":"2024-04-18T22:22:33.242905+0000","last_peered":"2024-04-18T22:22:33.242905+0000","last_clean":"2024-04-18T22:22:33.242905+0000","last_became_active":"2024-04-18T22:22:32.149253+0000","last_became_peered":"2024-04-18T22:22:32.149253+0000","last_unstale":"2024-04-18T22:22:33.242905+0000","last_undegraded":"2024-04-18T22:22:33.242905+0000","last_fullsized":"2024-04-18T22:22:33.242905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127402+0000","last_change":"2024-04-18T22:22:32.148298+0000","last_active":"2024-04-18T22:22:33.127402+0000","last_peered":"2024-04-18T22:22:33.127402+0000","last_clean":"2024-04-18T22:22:33.127402+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.127402+0000","last_undegraded":"2024-04-18T22:22:33.127402+0000","last_fullsized":"2024-04-18T22:22:33.127402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242310+0000","last_change":"2024-04-18T22:22:32.133832+0000","last_active":"2024-04-18T22:22:33.242310+0000","last_peered":"2024-04-18T22:22:33.242310+0000","last_clean":"2024-04-18T22:22:33.242310+0000","last_became_active":"2024-04-18T22:22:32.132844+0000","last_became_peered":"2024-04-18T22:22:32.132844+0000","last_unstale":"2024-04-18T22:22:33.242310+0000","last_undegraded":"2024-04-18T22:22:33.242310+0000","last_fullsized":"2024-04-18T22:22:33.242310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:04:32.092674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126733+0000","last_change":"2024-04-18T22:22:32.140365+0000","last_active":"2024-04-18T22:22:33.126733+0000","last_peered":"2024-04-18T22:22:33.126733+0000","last_clean":"2024-04-18T22:22:33.126733+0000","last_became_active":"2024-04-18T22:22:32.138147+0000","last_became_peered":"2024-04-18T22:22:32.138147+0000","last_unstale":"2024-04-18T22:22:33.126733+0000","last_undegraded":"2024-04-18T22:22:33.126733+0000","last_fullsized":"2024-04-18T22:22:33.126733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127632+0000","last_change":"2024-04-18T22:22:32.146043+0000","last_active":"2024-04-18T22:22:33.127632+0000","last_peered":"2024-04-18T22:22:33.127632+0000","last_clean":"2024-04-18T22:22:33.127632+0000","last_became_active":"2024-04-18T22:22:32.144174+0000","last_became_peered":"2024-04-18T22:22:32.144174+0000","last_unstale":"2024-04-18T22:22:33.127632+0000","last_undegraded":"2024-04-18T22:22:33.127632+0000","last_fullsized":"2024-04-18T22:22:33.127632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240784+0000","last_change":"2024-04-18T22:22:32.142740+0000","last_active":"2024-04-18T22:22:33.240784+0000","last_peered":"2024-04-18T22:22:33.240784+0000","last_clean":"2024-04-18T22:22:33.240784+0000","last_became_active":"2024-04-18T22:22:32.142048+0000","last_became_peered":"2024-04-18T22:22:32.142048+0000","last_unstale":"2024-04-18T22:22:33.240784+0000","last_undegraded":"2024-04-18T22:22:33.240784+0000","last_fullsized":"2024-04-18T22:22:33.240784+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240758+0000","last_change":"2024-04-18T22:22:32.129635+0000","last_active":"2024-04-18T22:22:33.240758+0000","last_peered":"2024-04-18T22:22:33.240758+0000","last_clean":"2024-04-18T22:22:33.240758+0000","last_became_active":"2024-04-18T22:22:32.128909+0000","last_became_peered":"2024-04-18T22:22:32.128909+0000","last_unstale":"2024-04-18T22:22:33.240758+0000","last_undegraded":"2024-04-18T22:22:33.240758+0000","last_fullsized":"2024-04-18T22:22:33.240758+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240016+0000","last_change":"2024-04-18T22:22:32.135755+0000","last_active":"2024-04-18T22:22:33.240016+0000","last_peered":"2024-04-18T22:22:33.240016+0000","last_clean":"2024-04-18T22:22:33.240016+0000","last_became_active":"2024-04-18T22:22:32.134588+0000","last_became_peered":"2024-04-18T22:22:32.134588+0000","last_unstale":"2024-04-18T22:22:33.240016+0000","last_undegraded":"2024-04-18T22:22:33.240016+0000","last_fullsized":"2024-04-18T22:22:33.240016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244495+0000","last_change":"2024-04-18T22:22:32.150194+0000","last_active":"2024-04-18T22:22:33.244495+0000","last_peered":"2024-04-18T22:22:33.244495+0000","last_clean":"2024-04-18T22:22:33.244495+0000","last_became_active":"2024-04-18T22:22:32.147919+0000","last_became_peered":"2024-04-18T22:22:32.147919+0000","last_unstale":"2024-04-18T22:22:33.244495+0000","last_undegraded":"2024-04-18T22:22:33.244495+0000","last_fullsized":"2024-04-18T22:22:33.244495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127840+0000","last_change":"2024-04-18T22:22:32.147917+0000","last_active":"2024-04-18T22:22:33.127840+0000","last_peered":"2024-04-18T22:22:33.127840+0000","last_clean":"2024-04-18T22:22:33.127840+0000","last_became_active":"2024-04-18T22:22:32.145953+0000","last_became_peered":"2024-04-18T22:22:32.145953+0000","last_unstale":"2024-04-18T22:22:33.127840+0000","last_undegraded":"2024-04-18T22:22:33.127840+0000","last_fullsized":"2024-04-18T22:22:33.127840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242825+0000","last_change":"2024-04-18T22:22:32.154655+0000","last_active":"2024-04-18T22:22:33.242825+0000","last_peered":"2024-04-18T22:22:33.242825+0000","last_clean":"2024-04-18T22:22:33.242825+0000","last_became_active":"2024-04-18T22:22:32.153966+0000","last_became_peered":"2024-04-18T22:22:32.153966+0000","last_unstale":"2024-04-18T22:22:33.242825+0000","last_undegraded":"2024-04-18T22:22:33.242825+0000","last_fullsized":"2024-04-18T22:22:33.242825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241750+0000","last_change":"2024-04-18T22:22:32.144928+0000","last_active":"2024-04-18T22:22:33.241750+0000","last_peered":"2024-04-18T22:22:33.241750+0000","last_clean":"2024-04-18T22:22:33.241750+0000","last_became_active":"2024-04-18T22:22:32.143282+0000","last_became_peered":"2024-04-18T22:22:32.143282+0000","last_unstale":"2024-04-18T22:22:33.241750+0000","last_undegraded":"2024-04-18T22:22:33.241750+0000","last_fullsized":"2024-04-18T22:22:33.241750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242442+0000","last_change":"2024-04-18T22:22:32.135811+0000","last_active":"2024-04-18T22:22:33.242442+0000","last_peered":"2024-04-18T22:22:33.242442+0000","last_clean":"2024-04-18T22:22:33.242442+0000","last_became_active":"2024-04-18T22:22:32.133269+0000","last_became_peered":"2024-04-18T22:22:32.133269+0000","last_unstale":"2024-04-18T22:22:33.242442+0000","last_undegraded":"2024-04-18T22:22:33.242442+0000","last_fullsized":"2024-04-18T22:22:33.242442+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:54:20.135916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126145+0000","last_change":"2024-04-18T22:22:32.141500+0000","last_active":"2024-04-18T22:22:33.126145+0000","last_peered":"2024-04-18T22:22:33.126145+0000","last_clean":"2024-04-18T22:22:33.126145+0000","last_became_active":"2024-04-18T22:22:32.140691+0000","last_became_peered":"2024-04-18T22:22:32.140691+0000","last_unstale":"2024-04-18T22:22:33.126145+0000","last_undegraded":"2024-04-18T22:22:33.126145+0000","last_fullsized":"2024-04-18T22:22:33.126145+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240965+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:22:33.240965+0000","last_peered":"2024-04-18T22:22:33.240965+0000","last_clean":"2024-04-18T22:22:33.240965+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:22:33.240965+0000","last_undegraded":"2024-04-18T22:22:33.240965+0000","last_fullsized":"2024-04-18T22:22:33.240965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126203+0000","last_change":"2024-04-18T22:22:32.141378+0000","last_active":"2024-04-18T22:22:33.126203+0000","last_peered":"2024-04-18T22:22:33.126203+0000","last_clean":"2024-04-18T22:22:33.126203+0000","last_became_active":"2024-04-18T22:22:32.140448+0000","last_became_peered":"2024-04-18T22:22:32.140448+0000","last_unstale":"2024-04-18T22:22:33.126203+0000","last_undegraded":"2024-04-18T22:22:33.126203+0000","last_fullsized":"2024-04-18T22:22:33.126203+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128370+0000","last_change":"2024-04-18T22:22:32.138865+0000","last_active":"2024-04-18T22:22:33.128370+0000","last_peered":"2024-04-18T22:22:33.128370+0000","last_clean":"2024-04-18T22:22:33.128370+0000","last_became_active":"2024-04-18T22:22:32.137793+0000","last_became_peered":"2024-04-18T22:22:32.137793+0000","last_unstale":"2024-04-18T22:22:33.128370+0000","last_undegraded":"2024-04-18T22:22:33.128370+0000","last_fullsized":"2024-04-18T22:22:33.128370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243541+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:22:33.243541+0000","last_peered":"2024-04-18T22:22:33.243541+0000","last_clean":"2024-04-18T22:22:33.243541+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:22:33.243541+0000","last_undegraded":"2024-04-18T22:22:33.243541+0000","last_fullsized":"2024-04-18T22:22:33.243541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242810+0000","last_change":"2024-04-18T22:22:32.146390+0000","last_active":"2024-04-18T22:22:33.242810+0000","last_peered":"2024-04-18T22:22:33.242810+0000","last_clean":"2024-04-18T22:22:33.242810+0000","last_became_active":"2024-04-18T22:22:32.145054+0000","last_became_peered":"2024-04-18T22:22:32.145054+0000","last_unstale":"2024-04-18T22:22:33.242810+0000","last_undegraded":"2024-04-18T22:22:33.242810+0000","last_fullsized":"2024-04-18T22:22:33.242810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:07:24.823488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127808+0000","last_change":"2024-04-18T22:22:32.132408+0000","last_active":"2024-04-18T22:22:33.127808+0000","last_peered":"2024-04-18T22:22:33.127808+0000","last_clean":"2024-04-18T22:22:33.127808+0000","last_became_active":"2024-04-18T22:22:32.131044+0000","last_became_peered":"2024-04-18T22:22:32.131044+0000","last_unstale":"2024-04-18T22:22:33.127808+0000","last_undegraded":"2024-04-18T22:22:33.127808+0000","last_fullsized":"2024-04-18T22:22:33.127808+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242233+0000","last_change":"2024-04-18T22:22:32.148879+0000","last_active":"2024-04-18T22:22:33.242233+0000","last_peered":"2024-04-18T22:22:33.242233+0000","last_clean":"2024-04-18T22:22:33.242233+0000","last_became_active":"2024-04-18T22:22:32.148070+0000","last_became_peered":"2024-04-18T22:22:32.148070+0000","last_unstale":"2024-04-18T22:22:33.242233+0000","last_undegraded":"2024-04-18T22:22:33.242233+0000","last_fullsized":"2024-04-18T22:22:33.242233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126582+0000","last_change":"2024-04-18T22:22:32.141405+0000","last_active":"2024-04-18T22:22:33.126582+0000","last_peered":"2024-04-18T22:22:33.126582+0000","last_clean":"2024-04-18T22:22:33.126582+0000","last_became_active":"2024-04-18T22:22:32.140369+0000","last_became_peered":"2024-04-18T22:22:32.140369+0000","last_unstale":"2024-04-18T22:22:33.126582+0000","last_undegraded":"2024-04-18T22:22:33.126582+0000","last_fullsized":"2024-04-18T22:22:33.126582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241205+0000","last_change":"2024-04-18T22:22:32.149208+0000","last_active":"2024-04-18T22:22:33.241205+0000","last_peered":"2024-04-18T22:22:33.241205+0000","last_clean":"2024-04-18T22:22:33.241205+0000","last_became_active":"2024-04-18T22:22:32.148344+0000","last_became_peered":"2024-04-18T22:22:32.148344+0000","last_unstale":"2024-04-18T22:22:33.241205+0000","last_undegraded":"2024-04-18T22:22:33.241205+0000","last_fullsized":"2024-04-18T22:22:33.241205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241714+0000","last_change":"2024-04-18T22:22:32.136695+0000","last_active":"2024-04-18T22:22:33.241714+0000","last_peered":"2024-04-18T22:22:33.241714+0000","last_clean":"2024-04-18T22:22:33.241714+0000","last_became_active":"2024-04-18T22:22:32.135901+0000","last_became_peered":"2024-04-18T22:22:32.135901+0000","last_unstale":"2024-04-18T22:22:33.241714+0000","last_undegraded":"2024-04-18T22:22:33.241714+0000","last_fullsized":"2024-04-18T22:22:33.241714+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127706+0000","last_change":"2024-04-18T22:22:32.144946+0000","last_active":"2024-04-18T22:22:33.127706+0000","last_peered":"2024-04-18T22:22:33.127706+0000","last_clean":"2024-04-18T22:22:33.127706+0000","last_became_active":"2024-04-18T22:22:32.143032+0000","last_became_peered":"2024-04-18T22:22:32.143032+0000","last_unstale":"2024-04-18T22:22:33.127706+0000","last_undegraded":"2024-04-18T22:22:33.127706+0000","last_fullsized":"2024-04-18T22:22:33.127706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242608+0000","last_change":"2024-04-18T22:22:31.145383+0000","last_active":"2024-04-18T22:22:33.242608+0000","last_peered":"2024-04-18T22:22:33.242608+0000","last_clean":"2024-04-18T22:22:33.242608+0000","last_became_active":"2024-04-18T22:22:31.143178+0000","last_became_peered":"2024-04-18T22:22:31.143178+0000","last_unstale":"2024-04-18T22:22:33.242608+0000","last_undegraded":"2024-04-18T22:22:33.242608+0000","last_fullsized":"2024-04-18T22:22:33.242608+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128208+0000","last_change":"2024-04-18T22:22:32.146608+0000","last_active":"2024-04-18T22:22:33.128208+0000","last_peered":"2024-04-18T22:22:33.128208+0000","last_clean":"2024-04-18T22:22:33.128208+0000","last_became_active":"2024-04-18T22:22:32.145761+0000","last_became_peered":"2024-04-18T22:22:32.145761+0000","last_unstale":"2024-04-18T22:22:33.128208+0000","last_undegraded":"2024-04-18T22:22:33.128208+0000","last_fullsized":"2024-04-18T22:22:33.128208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242107+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:22:33.242107+0000","last_peered":"2024-04-18T22:22:33.242107+0000","last_clean":"2024-04-18T22:22:33.242107+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:22:33.242107+0000","last_undegraded":"2024-04-18T22:22:33.242107+0000","last_fullsized":"2024-04-18T22:22:33.242107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243862+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:22:33.243862+0000","last_peered":"2024-04-18T22:22:33.243862+0000","last_clean":"2024-04-18T22:22:33.243862+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:22:33.243862+0000","last_undegraded":"2024-04-18T22:22:33.243862+0000","last_fullsized":"2024-04-18T22:22:33.243862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126186+0000","last_change":"2024-04-18T22:22:31.136552+0000","last_active":"2024-04-18T22:22:33.126186+0000","last_peered":"2024-04-18T22:22:33.126186+0000","last_clean":"2024-04-18T22:22:33.126186+0000","last_became_active":"2024-04-18T22:22:31.134824+0000","last_became_peered":"2024-04-18T22:22:31.134824+0000","last_unstale":"2024-04-18T22:22:33.126186+0000","last_undegraded":"2024-04-18T22:22:33.126186+0000","last_fullsized":"2024-04-18T22:22:33.126186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:07:17.380852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243033+0000","last_change":"2024-04-18T22:22:32.145785+0000","last_active":"2024-04-18T22:22:33.243033+0000","last_peered":"2024-04-18T22:22:33.243033+0000","last_clean":"2024-04-18T22:22:33.243033+0000","last_became_active":"2024-04-18T22:22:32.145166+0000","last_became_peered":"2024-04-18T22:22:32.145166+0000","last_unstale":"2024-04-18T22:22:33.243033+0000","last_undegraded":"2024-04-18T22:22:33.243033+0000","last_fullsized":"2024-04-18T22:22:33.243033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240502+0000","last_change":"2024-04-18T22:22:31.144229+0000","last_active":"2024-04-18T22:22:33.240502+0000","last_peered":"2024-04-18T22:22:33.240502+0000","last_clean":"2024-04-18T22:22:33.240502+0000","last_became_active":"2024-04-18T22:22:31.143597+0000","last_became_peered":"2024-04-18T22:22:31.143597+0000","last_unstale":"2024-04-18T22:22:33.240502+0000","last_undegraded":"2024-04-18T22:22:33.240502+0000","last_fullsized":"2024-04-18T22:22:33.240502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242009+0000","last_change":"2024-04-18T22:22:31.144343+0000","last_active":"2024-04-18T22:22:33.242009+0000","last_peered":"2024-04-18T22:22:33.242009+0000","last_clean":"2024-04-18T22:22:33.242009+0000","last_became_active":"2024-04-18T22:22:31.143325+0000","last_became_peered":"2024-04-18T22:22:31.143325+0000","last_unstale":"2024-04-18T22:22:33.242009+0000","last_undegraded":"2024-04-18T22:22:33.242009+0000","last_fullsized":"2024-04-18T22:22:33.242009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125867+0000","last_change":"2024-04-18T22:22:32.137726+0000","last_active":"2024-04-18T22:22:33.125867+0000","last_peered":"2024-04-18T22:22:33.125867+0000","last_clean":"2024-04-18T22:22:33.125867+0000","last_became_active":"2024-04-18T22:22:32.135216+0000","last_became_peered":"2024-04-18T22:22:32.135216+0000","last_unstale":"2024-04-18T22:22:33.125867+0000","last_undegraded":"2024-04-18T22:22:33.125867+0000","last_fullsized":"2024-04-18T22:22:33.125867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:01:43.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128180+0000","last_change":"2024-04-18T22:22:31.131784+0000","last_active":"2024-04-18T22:22:33.128180+0000","last_peered":"2024-04-18T22:22:33.128180+0000","last_clean":"2024-04-18T22:22:33.128180+0000","last_became_active":"2024-04-18T22:22:31.131002+0000","last_became_peered":"2024-04-18T22:22:31.131002+0000","last_unstale":"2024-04-18T22:22:33.128180+0000","last_undegraded":"2024-04-18T22:22:33.128180+0000","last_fullsized":"2024-04-18T22:22:33.128180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125897+0000","last_change":"2024-04-18T22:22:32.145000+0000","last_active":"2024-04-18T22:22:33.125897+0000","last_peered":"2024-04-18T22:22:33.125897+0000","last_clean":"2024-04-18T22:22:33.125897+0000","last_became_active":"2024-04-18T22:22:32.143522+0000","last_became_peered":"2024-04-18T22:22:32.143522+0000","last_unstale":"2024-04-18T22:22:33.125897+0000","last_undegraded":"2024-04-18T22:22:33.125897+0000","last_fullsized":"2024-04-18T22:22:33.125897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:26.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.144234+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.143532+0000","last_became_peered":"2024-04-18T22:22:31.143532+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:41.528679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241436+0000","last_change":"2024-04-18T22:22:32.135717+0000","last_active":"2024-04-18T22:22:33.241436+0000","last_peered":"2024-04-18T22:22:33.241436+0000","last_clean":"2024-04-18T22:22:33.241436+0000","last_became_active":"2024-04-18T22:22:32.134770+0000","last_became_peered":"2024-04-18T22:22:32.134770+0000","last_unstale":"2024-04-18T22:22:33.241436+0000","last_undegraded":"2024-04-18T22:22:33.241436+0000","last_fullsized":"2024-04-18T22:22:33.241436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242840+0000","last_change":"2024-04-18T22:22:31.147439+0000","last_active":"2024-04-18T22:22:33.242840+0000","last_peered":"2024-04-18T22:22:33.242840+0000","last_clean":"2024-04-18T22:22:33.242840+0000","last_became_active":"2024-04-18T22:22:31.146264+0000","last_became_peered":"2024-04-18T22:22:31.146264+0000","last_unstale":"2024-04-18T22:22:33.242840+0000","last_undegraded":"2024-04-18T22:22:33.242840+0000","last_fullsized":"2024-04-18T22:22:33.242840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242813+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:22:33.242813+0000","last_peered":"2024-04-18T22:22:33.242813+0000","last_clean":"2024-04-18T22:22:33.242813+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:22:33.242813+0000","last_undegraded":"2024-04-18T22:22:33.242813+0000","last_fullsized":"2024-04-18T22:22:33.242813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242737+0000","last_change":"2024-04-18T22:22:31.140451+0000","last_active":"2024-04-18T22:22:33.242737+0000","last_peered":"2024-04-18T22:22:33.242737+0000","last_clean":"2024-04-18T22:22:33.242737+0000","last_became_active":"2024-04-18T22:22:31.139655+0000","last_became_peered":"2024-04-18T22:22:31.139655+0000","last_unstale":"2024-04-18T22:22:33.242737+0000","last_undegraded":"2024-04-18T22:22:33.242737+0000","last_fullsized":"2024-04-18T22:22:33.242737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242161+0000","last_change":"2024-04-18T22:22:32.145869+0000","last_active":"2024-04-18T22:22:33.242161+0000","last_peered":"2024-04-18T22:22:33.242161+0000","last_clean":"2024-04-18T22:22:33.242161+0000","last_became_active":"2024-04-18T22:22:32.144117+0000","last_became_peered":"2024-04-18T22:22:32.144117+0000","last_unstale":"2024-04-18T22:22:33.242161+0000","last_undegraded":"2024-04-18T22:22:33.242161+0000","last_fullsized":"2024-04-18T22:22:33.242161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:39.186356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242361+0000","last_change":"2024-04-18T22:22:31.143197+0000","last_active":"2024-04-18T22:22:33.242361+0000","last_peered":"2024-04-18T22:22:33.242361+0000","last_clean":"2024-04-18T22:22:33.242361+0000","last_became_active":"2024-04-18T22:22:31.142393+0000","last_became_peered":"2024-04-18T22:22:31.142393+0000","last_unstale":"2024-04-18T22:22:33.242361+0000","last_undegraded":"2024-04-18T22:22:33.242361+0000","last_fullsized":"2024-04-18T22:22:33.242361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243118+0000","last_change":"2024-04-18T22:22:32.147396+0000","last_active":"2024-04-18T22:22:33.243118+0000","last_peered":"2024-04-18T22:22:33.243118+0000","last_clean":"2024-04-18T22:22:33.243118+0000","last_became_active":"2024-04-18T22:22:32.146327+0000","last_became_peered":"2024-04-18T22:22:32.146327+0000","last_unstale":"2024-04-18T22:22:33.243118+0000","last_undegraded":"2024-04-18T22:22:33.243118+0000","last_fullsized":"2024-04-18T22:22:33.243118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:16:31.884436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126713+0000","last_change":"2024-04-18T22:22:31.135058+0000","last_active":"2024-04-18T22:22:33.126713+0000","last_peered":"2024-04-18T22:22:33.126713+0000","last_clean":"2024-04-18T22:22:33.126713+0000","last_became_active":"2024-04-18T22:22:31.134213+0000","last_became_peered":"2024-04-18T22:22:31.134213+0000","last_unstale":"2024-04-18T22:22:33.126713+0000","last_undegraded":"2024-04-18T22:22:33.126713+0000","last_fullsized":"2024-04-18T22:22:33.126713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:35.837015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241581+0000","last_change":"2024-04-18T22:22:32.144956+0000","last_active":"2024-04-18T22:22:33.241581+0000","last_peered":"2024-04-18T22:22:33.241581+0000","last_clean":"2024-04-18T22:22:33.241581+0000","last_became_active":"2024-04-18T22:22:32.144171+0000","last_became_peered":"2024-04-18T22:22:32.144171+0000","last_unstale":"2024-04-18T22:22:33.241581+0000","last_undegraded":"2024-04-18T22:22:33.241581+0000","last_fullsized":"2024-04-18T22:22:33.241581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126257+0000","last_change":"2024-04-18T22:22:31.129222+0000","last_active":"2024-04-18T22:22:33.126257+0000","last_peered":"2024-04-18T22:22:33.126257+0000","last_clean":"2024-04-18T22:22:33.126257+0000","last_became_active":"2024-04-18T22:22:31.128618+0000","last_became_peered":"2024-04-18T22:22:31.128618+0000","last_unstale":"2024-04-18T22:22:33.126257+0000","last_undegraded":"2024-04-18T22:22:33.126257+0000","last_fullsized":"2024-04-18T22:22:33.126257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:58:49.687595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243356+0000","last_change":"2024-04-18T22:22:32.150911+0000","last_active":"2024-04-18T22:22:33.243356+0000","last_peered":"2024-04-18T22:22:33.243356+0000","last_clean":"2024-04-18T22:22:33.243356+0000","last_became_active":"2024-04-18T22:22:32.149188+0000","last_became_peered":"2024-04-18T22:22:32.149188+0000","last_unstale":"2024-04-18T22:22:33.243356+0000","last_undegraded":"2024-04-18T22:22:33.243356+0000","last_fullsized":"2024-04-18T22:22:33.243356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":222,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242605+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:33.242605+0000","last_peered":"2024-04-18T22:22:33.242605+0000","last_clean":"2024-04-18T22:22:33.242605+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:33.242605+0000","last_undegraded":"2024-04-18T22:22:33.242605+0000","last_fullsized":"2024-04-18T22:22:33.242605+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242816+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:33.242816+0000","last_peered":"2024-04-18T22:22:33.242816+0000","last_clean":"2024-04-18T22:22:33.242816+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:33.242816+0000","last_undegraded":"2024-04-18T22:22:33.242816+0000","last_fullsized":"2024-04-18T22:22:33.242816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242531+0000","last_change":"2024-04-18T22:22:31.128030+0000","last_active":"2024-04-18T22:22:33.242531+0000","last_peered":"2024-04-18T22:22:33.242531+0000","last_clean":"2024-04-18T22:22:33.242531+0000","last_became_active":"2024-04-18T22:22:31.126939+0000","last_became_peered":"2024-04-18T22:22:31.126939+0000","last_unstale":"2024-04-18T22:22:33.242531+0000","last_undegraded":"2024-04-18T22:22:33.242531+0000","last_fullsized":"2024-04-18T22:22:33.242531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243540+0000","last_change":"2024-04-18T22:22:32.153417+0000","last_active":"2024-04-18T22:22:33.243540+0000","last_peered":"2024-04-18T22:22:33.243540+0000","last_clean":"2024-04-18T22:22:33.243540+0000","last_became_active":"2024-04-18T22:22:32.152653+0000","last_became_peered":"2024-04-18T22:22:32.152653+0000","last_unstale":"2024-04-18T22:22:33.243540+0000","last_undegraded":"2024-04-18T22:22:33.243540+0000","last_fullsized":"2024-04-18T22:22:33.243540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:49:12.269993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242307+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:33.242307+0000","last_peered":"2024-04-18T22:22:33.242307+0000","last_clean":"2024-04-18T22:22:33.242307+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:33.242307+0000","last_undegraded":"2024-04-18T22:22:33.242307+0000","last_fullsized":"2024-04-18T22:22:33.242307+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242019+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:22:33.242019+0000","last_peered":"2024-04-18T22:22:33.242019+0000","last_clean":"2024-04-18T22:22:33.242019+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:22:33.242019+0000","last_undegraded":"2024-04-18T22:22:33.242019+0000","last_fullsized":"2024-04-18T22:22:33.242019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127898+0000","last_change":"2024-04-18T22:22:32.147651+0000","last_active":"2024-04-18T22:22:33.127898+0000","last_peered":"2024-04-18T22:22:33.127898+0000","last_clean":"2024-04-18T22:22:33.127898+0000","last_became_active":"2024-04-18T22:22:32.143774+0000","last_became_peered":"2024-04-18T22:22:32.143774+0000","last_unstale":"2024-04-18T22:22:33.127898+0000","last_undegraded":"2024-04-18T22:22:33.127898+0000","last_fullsized":"2024-04-18T22:22:33.127898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242167+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:33.242167+0000","last_peered":"2024-04-18T22:22:33.242167+0000","last_clean":"2024-04-18T22:22:33.242167+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:33.242167+0000","last_undegraded":"2024-04-18T22:22:33.242167+0000","last_fullsized":"2024-04-18T22:22:33.242167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126710+0000","last_change":"2024-04-18T22:22:31.132958+0000","last_active":"2024-04-18T22:22:33.126710+0000","last_peered":"2024-04-18T22:22:33.126710+0000","last_clean":"2024-04-18T22:22:33.126710+0000","last_became_active":"2024-04-18T22:22:31.132186+0000","last_became_peered":"2024-04-18T22:22:31.132186+0000","last_unstale":"2024-04-18T22:22:33.126710+0000","last_undegraded":"2024-04-18T22:22:33.126710+0000","last_fullsized":"2024-04-18T22:22:33.126710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243413+0000","last_change":"2024-04-18T22:22:32.150208+0000","last_active":"2024-04-18T22:22:33.243413+0000","last_peered":"2024-04-18T22:22:33.243413+0000","last_clean":"2024-04-18T22:22:33.243413+0000","last_became_active":"2024-04-18T22:22:32.148155+0000","last_became_peered":"2024-04-18T22:22:32.148155+0000","last_unstale":"2024-04-18T22:22:33.243413+0000","last_undegraded":"2024-04-18T22:22:33.243413+0000","last_fullsized":"2024-04-18T22:22:33.243413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241760+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:33.241760+0000","last_peered":"2024-04-18T22:22:33.241760+0000","last_clean":"2024-04-18T22:22:33.241760+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:33.241760+0000","last_undegraded":"2024-04-18T22:22:33.241760+0000","last_fullsized":"2024-04-18T22:22:33.241760+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242877+0000","last_change":"2024-04-18T22:22:31.146898+0000","last_active":"2024-04-18T22:22:33.242877+0000","last_peered":"2024-04-18T22:22:33.242877+0000","last_clean":"2024-04-18T22:22:33.242877+0000","last_became_active":"2024-04-18T22:22:31.146071+0000","last_became_peered":"2024-04-18T22:22:31.146071+0000","last_unstale":"2024-04-18T22:22:33.242877+0000","last_undegraded":"2024-04-18T22:22:33.242877+0000","last_fullsized":"2024-04-18T22:22:33.242877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243542+0000","last_change":"2024-04-18T22:22:32.147890+0000","last_active":"2024-04-18T22:22:33.243542+0000","last_peered":"2024-04-18T22:22:33.243542+0000","last_clean":"2024-04-18T22:22:33.243542+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.243542+0000","last_undegraded":"2024-04-18T22:22:33.243542+0000","last_fullsized":"2024-04-18T22:22:33.243542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243666+0000","last_change":"2024-04-18T22:22:31.146681+0000","last_active":"2024-04-18T22:22:33.243666+0000","last_peered":"2024-04-18T22:22:33.243666+0000","last_clean":"2024-04-18T22:22:33.243666+0000","last_became_active":"2024-04-18T22:22:31.145764+0000","last_became_peered":"2024-04-18T22:22:31.145764+0000","last_unstale":"2024-04-18T22:22:33.243666+0000","last_undegraded":"2024-04-18T22:22:33.243666+0000","last_fullsized":"2024-04-18T22:22:33.243666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:33.679846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243510+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:33.243510+0000","last_peered":"2024-04-18T22:22:33.243510+0000","last_clean":"2024-04-18T22:22:33.243510+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:33.243510+0000","last_undegraded":"2024-04-18T22:22:33.243510+0000","last_fullsized":"2024-04-18T22:22:33.243510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127522+0000","last_change":"2024-04-18T22:22:32.127971+0000","last_active":"2024-04-18T22:22:33.127522+0000","last_peered":"2024-04-18T22:22:33.127522+0000","last_clean":"2024-04-18T22:22:33.127522+0000","last_became_active":"2024-04-18T22:22:32.126518+0000","last_became_peered":"2024-04-18T22:22:32.126518+0000","last_unstale":"2024-04-18T22:22:33.127522+0000","last_undegraded":"2024-04-18T22:22:33.127522+0000","last_fullsized":"2024-04-18T22:22:33.127522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243415+0000","last_change":"2024-04-18T22:22:31.127787+0000","last_active":"2024-04-18T22:22:33.243415+0000","last_peered":"2024-04-18T22:22:33.243415+0000","last_clean":"2024-04-18T22:22:33.243415+0000","last_became_active":"2024-04-18T22:22:31.127173+0000","last_became_peered":"2024-04-18T22:22:31.127173+0000","last_unstale":"2024-04-18T22:22:33.243415+0000","last_undegraded":"2024-04-18T22:22:33.243415+0000","last_fullsized":"2024-04-18T22:22:33.243415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128505+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:33.128505+0000","last_peered":"2024-04-18T22:22:33.128505+0000","last_clean":"2024-04-18T22:22:33.128505+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:33.128505+0000","last_undegraded":"2024-04-18T22:22:33.128505+0000","last_fullsized":"2024-04-18T22:22:33.128505+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126910+0000","last_change":"2024-04-18T22:22:32.136161+0000","last_active":"2024-04-18T22:22:33.126910+0000","last_peered":"2024-04-18T22:22:33.126910+0000","last_clean":"2024-04-18T22:22:33.126910+0000","last_became_active":"2024-04-18T22:22:32.135253+0000","last_became_peered":"2024-04-18T22:22:32.135253+0000","last_unstale":"2024-04-18T22:22:33.126910+0000","last_undegraded":"2024-04-18T22:22:33.126910+0000","last_fullsized":"2024-04-18T22:22:33.126910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244171+0000","last_change":"2024-04-18T22:22:31.136319+0000","last_active":"2024-04-18T22:22:33.244171+0000","last_peered":"2024-04-18T22:22:33.244171+0000","last_clean":"2024-04-18T22:22:33.244171+0000","last_became_active":"2024-04-18T22:22:31.135689+0000","last_became_peered":"2024-04-18T22:22:31.135689+0000","last_unstale":"2024-04-18T22:22:33.244171+0000","last_undegraded":"2024-04-18T22:22:33.244171+0000","last_fullsized":"2024-04-18T22:22:33.244171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244003+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:33.244003+0000","last_peered":"2024-04-18T22:22:33.244003+0000","last_clean":"2024-04-18T22:22:33.244003+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:33.244003+0000","last_undegraded":"2024-04-18T22:22:33.244003+0000","last_fullsized":"2024-04-18T22:22:33.244003+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242936+0000","last_change":"2024-04-18T22:22:32.144432+0000","last_active":"2024-04-18T22:22:33.242936+0000","last_peered":"2024-04-18T22:22:33.242936+0000","last_clean":"2024-04-18T22:22:33.242936+0000","last_became_active":"2024-04-18T22:22:32.142947+0000","last_became_peered":"2024-04-18T22:22:32.142947+0000","last_unstale":"2024-04-18T22:22:33.242936+0000","last_undegraded":"2024-04-18T22:22:33.242936+0000","last_fullsized":"2024-04-18T22:22:33.242936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:34.119061+0000","last_change":"2024-04-18T22:22:31.143202+0000","last_active":"2024-04-18T22:22:34.119061+0000","last_peered":"2024-04-18T22:22:34.119061+0000","last_clean":"2024-04-18T22:22:34.119061+0000","last_became_active":"2024-04-18T22:22:31.142465+0000","last_became_peered":"2024-04-18T22:22:31.142465+0000","last_unstale":"2024-04-18T22:22:34.119061+0000","last_undegraded":"2024-04-18T22:22:34.119061+0000","last_fullsized":"2024-04-18T22:22:34.119061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241961+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:33.241961+0000","last_peered":"2024-04-18T22:22:33.241961+0000","last_clean":"2024-04-18T22:22:33.241961+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:33.241961+0000","last_undegraded":"2024-04-18T22:22:33.241961+0000","last_fullsized":"2024-04-18T22:22:33.241961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125912+0000","last_change":"2024-04-18T22:22:32.132968+0000","last_active":"2024-04-18T22:22:33.125912+0000","last_peered":"2024-04-18T22:22:33.125912+0000","last_clean":"2024-04-18T22:22:33.125912+0000","last_became_active":"2024-04-18T22:22:32.131596+0000","last_became_peered":"2024-04-18T22:22:32.131596+0000","last_unstale":"2024-04-18T22:22:33.125912+0000","last_undegraded":"2024-04-18T22:22:33.125912+0000","last_fullsized":"2024-04-18T22:22:33.125912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241097+0000","last_change":"2024-04-18T22:22:32.145633+0000","last_active":"2024-04-18T22:22:33.241097+0000","last_peered":"2024-04-18T22:22:33.241097+0000","last_clean":"2024-04-18T22:22:33.241097+0000","last_became_active":"2024-04-18T22:22:32.142868+0000","last_became_peered":"2024-04-18T22:22:32.142868+0000","last_unstale":"2024-04-18T22:22:33.241097+0000","last_undegraded":"2024-04-18T22:22:33.241097+0000","last_fullsized":"2024-04-18T22:22:33.241097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242663+0000","last_change":"2024-04-18T22:22:31.144413+0000","last_active":"2024-04-18T22:22:33.242663+0000","last_peered":"2024-04-18T22:22:33.242663+0000","last_clean":"2024-04-18T22:22:33.242663+0000","last_became_active":"2024-04-18T22:22:31.143704+0000","last_became_peered":"2024-04-18T22:22:31.143704+0000","last_unstale":"2024-04-18T22:22:33.242663+0000","last_undegraded":"2024-04-18T22:22:33.242663+0000","last_fullsized":"2024-04-18T22:22:33.242663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241992+0000","last_change":"2024-04-18T22:22:32.148407+0000","last_active":"2024-04-18T22:22:33.241992+0000","last_peered":"2024-04-18T22:22:33.241992+0000","last_clean":"2024-04-18T22:22:33.241992+0000","last_became_active":"2024-04-18T22:22:32.147033+0000","last_became_peered":"2024-04-18T22:22:32.147033+0000","last_unstale":"2024-04-18T22:22:33.241992+0000","last_undegraded":"2024-04-18T22:22:33.241992+0000","last_fullsized":"2024-04-18T22:22:33.241992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128409+0000","last_change":"2024-04-18T22:22:31.133306+0000","last_active":"2024-04-18T22:22:33.128409+0000","last_peered":"2024-04-18T22:22:33.128409+0000","last_clean":"2024-04-18T22:22:33.128409+0000","last_became_active":"2024-04-18T22:22:31.132648+0000","last_became_peered":"2024-04-18T22:22:31.132648+0000","last_unstale":"2024-04-18T22:22:33.128409+0000","last_undegraded":"2024-04-18T22:22:33.128409+0000","last_fullsized":"2024-04-18T22:22:33.128409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242551+0000","last_change":"2024-04-18T22:22:31.146827+0000","last_active":"2024-04-18T22:22:33.242551+0000","last_peered":"2024-04-18T22:22:33.242551+0000","last_clean":"2024-04-18T22:22:33.242551+0000","last_became_active":"2024-04-18T22:22:31.146063+0000","last_became_peered":"2024-04-18T22:22:31.146063+0000","last_unstale":"2024-04-18T22:22:33.242551+0000","last_undegraded":"2024-04-18T22:22:33.242551+0000","last_fullsized":"2024-04-18T22:22:33.242551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241955+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:22:33.241955+0000","last_peered":"2024-04-18T22:22:33.241955+0000","last_clean":"2024-04-18T22:22:33.241955+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:22:33.241955+0000","last_undegraded":"2024-04-18T22:22:33.241955+0000","last_fullsized":"2024-04-18T22:22:33.241955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243856+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:22:33.243856+0000","last_peered":"2024-04-18T22:22:33.243856+0000","last_clean":"2024-04-18T22:22:33.243856+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:22:33.243856+0000","last_undegraded":"2024-04-18T22:22:33.243856+0000","last_fullsized":"2024-04-18T22:22:33.243856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241179+0000","last_change":"2024-04-18T22:22:32.146927+0000","last_active":"2024-04-18T22:22:33.241179+0000","last_peered":"2024-04-18T22:22:33.241179+0000","last_clean":"2024-04-18T22:22:33.241179+0000","last_became_active":"2024-04-18T22:22:32.146040+0000","last_became_peered":"2024-04-18T22:22:32.146040+0000","last_unstale":"2024-04-18T22:22:33.241179+0000","last_undegraded":"2024-04-18T22:22:33.241179+0000","last_fullsized":"2024-04-18T22:22:33.241179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128951+0000","last_change":"2024-04-18T22:22:32.139063+0000","last_active":"2024-04-18T22:22:33.128951+0000","last_peered":"2024-04-18T22:22:33.128951+0000","last_clean":"2024-04-18T22:22:33.128951+0000","last_became_active":"2024-04-18T22:22:32.138016+0000","last_became_peered":"2024-04-18T22:22:32.138016+0000","last_unstale":"2024-04-18T22:22:33.128951+0000","last_undegraded":"2024-04-18T22:22:33.128951+0000","last_fullsized":"2024-04-18T22:22:33.128951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241615+0000","last_change":"2024-04-18T22:22:31.130452+0000","last_active":"2024-04-18T22:22:33.241615+0000","last_peered":"2024-04-18T22:22:33.241615+0000","last_clean":"2024-04-18T22:22:33.241615+0000","last_became_active":"2024-04-18T22:22:31.129555+0000","last_became_peered":"2024-04-18T22:22:31.129555+0000","last_unstale":"2024-04-18T22:22:33.241615+0000","last_undegraded":"2024-04-18T22:22:33.241615+0000","last_fullsized":"2024-04-18T22:22:33.241615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243327+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:22:33.243327+0000","last_peered":"2024-04-18T22:22:33.243327+0000","last_clean":"2024-04-18T22:22:33.243327+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:22:33.243327+0000","last_undegraded":"2024-04-18T22:22:33.243327+0000","last_fullsized":"2024-04-18T22:22:33.243327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241696+0000","last_change":"2024-04-18T22:22:31.133021+0000","last_active":"2024-04-18T22:22:33.241696+0000","last_peered":"2024-04-18T22:22:33.241696+0000","last_clean":"2024-04-18T22:22:33.241696+0000","last_became_active":"2024-04-18T22:22:31.132544+0000","last_became_peered":"2024-04-18T22:22:31.132544+0000","last_unstale":"2024-04-18T22:22:33.241696+0000","last_undegraded":"2024-04-18T22:22:33.241696+0000","last_fullsized":"2024-04-18T22:22:33.241696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127774+0000","last_change":"2024-04-18T22:22:32.138696+0000","last_active":"2024-04-18T22:22:33.127774+0000","last_peered":"2024-04-18T22:22:33.127774+0000","last_clean":"2024-04-18T22:22:33.127774+0000","last_became_active":"2024-04-18T22:22:32.137900+0000","last_became_peered":"2024-04-18T22:22:32.137900+0000","last_unstale":"2024-04-18T22:22:33.127774+0000","last_undegraded":"2024-04-18T22:22:33.127774+0000","last_fullsized":"2024-04-18T22:22:33.127774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126866+0000","last_change":"2024-04-18T22:22:31.133358+0000","last_active":"2024-04-18T22:22:33.126866+0000","last_peered":"2024-04-18T22:22:33.126866+0000","last_clean":"2024-04-18T22:22:33.126866+0000","last_became_active":"2024-04-18T22:22:31.132642+0000","last_became_peered":"2024-04-18T22:22:31.132642+0000","last_unstale":"2024-04-18T22:22:33.126866+0000","last_undegraded":"2024-04-18T22:22:33.126866+0000","last_fullsized":"2024-04-18T22:22:33.126866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127236+0000","last_change":"2024-04-18T22:22:32.146126+0000","last_active":"2024-04-18T22:22:33.127236+0000","last_peered":"2024-04-18T22:22:33.127236+0000","last_clean":"2024-04-18T22:22:33.127236+0000","last_became_active":"2024-04-18T22:22:32.144083+0000","last_became_peered":"2024-04-18T22:22:32.144083+0000","last_unstale":"2024-04-18T22:22:33.127236+0000","last_undegraded":"2024-04-18T22:22:33.127236+0000","last_fullsized":"2024-04-18T22:22:33.127236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127648+0000","last_change":"2024-04-18T22:22:31.136636+0000","last_active":"2024-04-18T22:22:33.127648+0000","last_peered":"2024-04-18T22:22:33.127648+0000","last_clean":"2024-04-18T22:22:33.127648+0000","last_became_active":"2024-04-18T22:22:31.135039+0000","last_became_peered":"2024-04-18T22:22:31.135039+0000","last_unstale":"2024-04-18T22:22:33.127648+0000","last_undegraded":"2024-04-18T22:22:33.127648+0000","last_fullsized":"2024-04-18T22:22:33.127648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.293851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240793+0000","last_change":"2024-04-18T22:22:32.134794+0000","last_active":"2024-04-18T22:22:33.240793+0000","last_peered":"2024-04-18T22:22:33.240793+0000","last_clean":"2024-04-18T22:22:33.240793+0000","last_became_active":"2024-04-18T22:22:32.133858+0000","last_became_peered":"2024-04-18T22:22:32.133858+0000","last_unstale":"2024-04-18T22:22:33.240793+0000","last_undegraded":"2024-04-18T22:22:33.240793+0000","last_fullsized":"2024-04-18T22:22:33.240793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242444+0000","last_change":"2024-04-18T22:22:31.147456+0000","last_active":"2024-04-18T22:22:33.242444+0000","last_peered":"2024-04-18T22:22:33.242444+0000","last_clean":"2024-04-18T22:22:33.242444+0000","last_became_active":"2024-04-18T22:22:31.145624+0000","last_became_peered":"2024-04-18T22:22:31.145624+0000","last_unstale":"2024-04-18T22:22:33.242444+0000","last_undegraded":"2024-04-18T22:22:33.242444+0000","last_fullsized":"2024-04-18T22:22:33.242444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241579+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.241579+0000","last_peered":"2024-04-18T22:22:33.241579+0000","last_clean":"2024-04-18T22:22:33.241579+0000","last_became_active":"2024-04-18T22:22:32.143184+0000","last_became_peered":"2024-04-18T22:22:32.143184+0000","last_unstale":"2024-04-18T22:22:33.241579+0000","last_undegraded":"2024-04-18T22:22:33.241579+0000","last_fullsized":"2024-04-18T22:22:33.241579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128174+0000","last_change":"2024-04-18T22:22:31.138492+0000","last_active":"2024-04-18T22:22:33.128174+0000","last_peered":"2024-04-18T22:22:33.128174+0000","last_clean":"2024-04-18T22:22:33.128174+0000","last_became_active":"2024-04-18T22:22:31.134646+0000","last_became_peered":"2024-04-18T22:22:31.134646+0000","last_unstale":"2024-04-18T22:22:33.128174+0000","last_undegraded":"2024-04-18T22:22:33.128174+0000","last_fullsized":"2024-04-18T22:22:33.128174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127146+0000","last_change":"2024-04-18T22:22:32.137473+0000","last_active":"2024-04-18T22:22:33.127146+0000","last_peered":"2024-04-18T22:22:33.127146+0000","last_clean":"2024-04-18T22:22:33.127146+0000","last_became_active":"2024-04-18T22:22:32.136025+0000","last_became_peered":"2024-04-18T22:22:32.136025+0000","last_unstale":"2024-04-18T22:22:33.127146+0000","last_undegraded":"2024-04-18T22:22:33.127146+0000","last_fullsized":"2024-04-18T22:22:33.127146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243824+0000","last_change":"2024-04-18T22:22:31.130827+0000","last_active":"2024-04-18T22:22:33.243824+0000","last_peered":"2024-04-18T22:22:33.243824+0000","last_clean":"2024-04-18T22:22:33.243824+0000","last_became_active":"2024-04-18T22:22:31.129728+0000","last_became_peered":"2024-04-18T22:22:31.129728+0000","last_unstale":"2024-04-18T22:22:33.243824+0000","last_undegraded":"2024-04-18T22:22:33.243824+0000","last_fullsized":"2024-04-18T22:22:33.243824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242152+0000","last_change":"2024-04-18T22:22:32.144325+0000","last_active":"2024-04-18T22:22:33.242152+0000","last_peered":"2024-04-18T22:22:33.242152+0000","last_clean":"2024-04-18T22:22:33.242152+0000","last_became_active":"2024-04-18T22:22:32.143495+0000","last_became_peered":"2024-04-18T22:22:32.143495+0000","last_unstale":"2024-04-18T22:22:33.242152+0000","last_undegraded":"2024-04-18T22:22:33.242152+0000","last_fullsized":"2024-04-18T22:22:33.242152+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.142640+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.141819+0000","last_became_peered":"2024-04-18T22:22:31.141819+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":245760,"data_stored":647890,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":647274,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":55728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:39.917 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-18T22:22:39.917 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-18T22:22:39.917 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-18T22:22:39.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-18T22:22:39.918 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-18T22:22:40.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:40.321 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4183746619},{"type":"v1","addr":"172.21.15.32:6801","nonce":4183746619}]},"active_addr":"172.21.15.32:6801/4183746619","active_change":"2024-04-18T22:22:14.817140+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1066757394}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1049496302}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":323556892}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1051001453}]}]} 2024-04-18T22:22:40.324 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-18T22:22:40.325 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-18T22:22:40.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:40.620 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:40.621 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:33.115398+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:40.639 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-18T22:22:40.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:40.939 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:33.115398+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:40.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-18T22:22:40.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-18T22:22:40.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-18T22:22:41.108 INFO:teuthology.orchestra.run.smithi032.stdout:34359738374 2024-04-18T22:22:41.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T22:22:41.120 INFO:teuthology.orchestra.run.smithi032.stdout:34359738376 2024-04-18T22:22:41.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T22:22:41.123 INFO:teuthology.orchestra.run.smithi032.stdout:34359738376 2024-04-18T22:22:41.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T22:22:41.400 INFO:teuthology.orchestra.run.smithi032.stdout:34359738373 2024-04-18T22:22:41.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.0 2024-04-18T22:22:41.444 INFO:teuthology.orchestra.run.smithi032.stdout:34359738375 2024-04-18T22:22:41.444 INFO:teuthology.orchestra.run.smithi032.stdout:34359738375 2024-04-18T22:22:41.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738375 for osd.2 2024-04-18T22:22:41.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738375 for osd.1 2024-04-18T22:22:42.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T22:22:42.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T22:22:42.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T22:22:42.734 INFO:teuthology.orchestra.run.smithi032.stdout:34359738373 2024-04-18T22:22:42.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.0 2024-04-18T22:22:42.760 INFO:teuthology.orchestra.run.smithi032.stdout:34359738375 2024-04-18T22:22:42.777 INFO:teuthology.orchestra.run.smithi032.stdout:34359738375 2024-04-18T22:22:42.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738375 for osd.2 2024-04-18T22:22:42.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738375 for osd.1 2024-04-18T22:22:43.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T22:22:43.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T22:22:43.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T22:22:44.069 INFO:teuthology.orchestra.run.smithi032.stdout:34359738374 2024-04-18T22:22:44.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.0 2024-04-18T22:22:44.086 DEBUG:teuthology.parallel:result is None 2024-04-18T22:22:44.096 INFO:teuthology.orchestra.run.smithi032.stdout:34359738376 2024-04-18T22:22:44.096 INFO:teuthology.orchestra.run.smithi032.stdout:34359738376 2024-04-18T22:22:44.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738376 for osd.2 2024-04-18T22:22:44.122 DEBUG:teuthology.parallel:result is None 2024-04-18T22:22:44.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738376 for osd.1 2024-04-18T22:22:44.122 DEBUG:teuthology.parallel:result is None 2024-04-18T22:22:44.122 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-18T22:22:44.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:44.459 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:44.461 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:44.476 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-18T22:22:42.831636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":90020,"kb_used_data":632,"kb_used_omap":4,"kb_used_meta":89339,"kb_avail":283025500,"statfs":{"total":289910292480,"available":289818112000,"internally_reserved":0,"allocated":647168,"data_stored":1350892,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":91483471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003165"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128432+0000","last_change":"2024-04-18T22:22:32.145997+0000","last_active":"2024-04-18T22:22:33.128432+0000","last_peered":"2024-04-18T22:22:33.128432+0000","last_clean":"2024-04-18T22:22:33.128432+0000","last_became_active":"2024-04-18T22:22:32.143852+0000","last_became_peered":"2024-04-18T22:22:32.143852+0000","last_unstale":"2024-04-18T22:22:33.128432+0000","last_undegraded":"2024-04-18T22:22:33.128432+0000","last_fullsized":"2024-04-18T22:22:33.128432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126921+0000","last_change":"2024-04-18T22:22:32.148788+0000","last_active":"2024-04-18T22:22:33.126921+0000","last_peered":"2024-04-18T22:22:33.126921+0000","last_clean":"2024-04-18T22:22:33.126921+0000","last_became_active":"2024-04-18T22:22:32.146781+0000","last_became_peered":"2024-04-18T22:22:32.146781+0000","last_unstale":"2024-04-18T22:22:33.126921+0000","last_undegraded":"2024-04-18T22:22:33.126921+0000","last_fullsized":"2024-04-18T22:22:33.126921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241643+0000","last_change":"2024-04-18T22:22:32.152874+0000","last_active":"2024-04-18T22:22:33.241643+0000","last_peered":"2024-04-18T22:22:33.241643+0000","last_clean":"2024-04-18T22:22:33.241643+0000","last_became_active":"2024-04-18T22:22:32.152154+0000","last_became_peered":"2024-04-18T22:22:32.152154+0000","last_unstale":"2024-04-18T22:22:33.241643+0000","last_undegraded":"2024-04-18T22:22:33.241643+0000","last_fullsized":"2024-04-18T22:22:33.241643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242573+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.242573+0000","last_peered":"2024-04-18T22:22:33.242573+0000","last_clean":"2024-04-18T22:22:33.242573+0000","last_became_active":"2024-04-18T22:22:32.142591+0000","last_became_peered":"2024-04-18T22:22:32.142591+0000","last_unstale":"2024-04-18T22:22:33.242573+0000","last_undegraded":"2024-04-18T22:22:33.242573+0000","last_fullsized":"2024-04-18T22:22:33.242573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:24:32.079309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127358+0000","last_change":"2024-04-18T22:22:32.136433+0000","last_active":"2024-04-18T22:22:33.127358+0000","last_peered":"2024-04-18T22:22:33.127358+0000","last_clean":"2024-04-18T22:22:33.127358+0000","last_became_active":"2024-04-18T22:22:32.135586+0000","last_became_peered":"2024-04-18T22:22:32.135586+0000","last_unstale":"2024-04-18T22:22:33.127358+0000","last_undegraded":"2024-04-18T22:22:33.127358+0000","last_fullsized":"2024-04-18T22:22:33.127358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240606+0000","last_change":"2024-04-18T22:22:32.134957+0000","last_active":"2024-04-18T22:22:33.240606+0000","last_peered":"2024-04-18T22:22:33.240606+0000","last_clean":"2024-04-18T22:22:33.240606+0000","last_became_active":"2024-04-18T22:22:32.133797+0000","last_became_peered":"2024-04-18T22:22:32.133797+0000","last_unstale":"2024-04-18T22:22:33.240606+0000","last_undegraded":"2024-04-18T22:22:33.240606+0000","last_fullsized":"2024-04-18T22:22:33.240606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240547+0000","last_change":"2024-04-18T22:22:32.133775+0000","last_active":"2024-04-18T22:22:33.240547+0000","last_peered":"2024-04-18T22:22:33.240547+0000","last_clean":"2024-04-18T22:22:33.240547+0000","last_became_active":"2024-04-18T22:22:32.132972+0000","last_became_peered":"2024-04-18T22:22:32.132972+0000","last_unstale":"2024-04-18T22:22:33.240547+0000","last_undegraded":"2024-04-18T22:22:33.240547+0000","last_fullsized":"2024-04-18T22:22:33.240547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.239986+0000","last_change":"2024-04-18T22:22:32.136411+0000","last_active":"2024-04-18T22:22:33.239986+0000","last_peered":"2024-04-18T22:22:33.239986+0000","last_clean":"2024-04-18T22:22:33.239986+0000","last_became_active":"2024-04-18T22:22:32.132809+0000","last_became_peered":"2024-04-18T22:22:32.132809+0000","last_unstale":"2024-04-18T22:22:33.239986+0000","last_undegraded":"2024-04-18T22:22:33.239986+0000","last_fullsized":"2024-04-18T22:22:33.239986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242905+0000","last_change":"2024-04-18T22:22:32.150575+0000","last_active":"2024-04-18T22:22:33.242905+0000","last_peered":"2024-04-18T22:22:33.242905+0000","last_clean":"2024-04-18T22:22:33.242905+0000","last_became_active":"2024-04-18T22:22:32.149253+0000","last_became_peered":"2024-04-18T22:22:32.149253+0000","last_unstale":"2024-04-18T22:22:33.242905+0000","last_undegraded":"2024-04-18T22:22:33.242905+0000","last_fullsized":"2024-04-18T22:22:33.242905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127402+0000","last_change":"2024-04-18T22:22:32.148298+0000","last_active":"2024-04-18T22:22:33.127402+0000","last_peered":"2024-04-18T22:22:33.127402+0000","last_clean":"2024-04-18T22:22:33.127402+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.127402+0000","last_undegraded":"2024-04-18T22:22:33.127402+0000","last_fullsized":"2024-04-18T22:22:33.127402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242310+0000","last_change":"2024-04-18T22:22:32.133832+0000","last_active":"2024-04-18T22:22:33.242310+0000","last_peered":"2024-04-18T22:22:33.242310+0000","last_clean":"2024-04-18T22:22:33.242310+0000","last_became_active":"2024-04-18T22:22:32.132844+0000","last_became_peered":"2024-04-18T22:22:32.132844+0000","last_unstale":"2024-04-18T22:22:33.242310+0000","last_undegraded":"2024-04-18T22:22:33.242310+0000","last_fullsized":"2024-04-18T22:22:33.242310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:04:32.092674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126733+0000","last_change":"2024-04-18T22:22:32.140365+0000","last_active":"2024-04-18T22:22:33.126733+0000","last_peered":"2024-04-18T22:22:33.126733+0000","last_clean":"2024-04-18T22:22:33.126733+0000","last_became_active":"2024-04-18T22:22:32.138147+0000","last_became_peered":"2024-04-18T22:22:32.138147+0000","last_unstale":"2024-04-18T22:22:33.126733+0000","last_undegraded":"2024-04-18T22:22:33.126733+0000","last_fullsized":"2024-04-18T22:22:33.126733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127632+0000","last_change":"2024-04-18T22:22:32.146043+0000","last_active":"2024-04-18T22:22:33.127632+0000","last_peered":"2024-04-18T22:22:33.127632+0000","last_clean":"2024-04-18T22:22:33.127632+0000","last_became_active":"2024-04-18T22:22:32.144174+0000","last_became_peered":"2024-04-18T22:22:32.144174+0000","last_unstale":"2024-04-18T22:22:33.127632+0000","last_undegraded":"2024-04-18T22:22:33.127632+0000","last_fullsized":"2024-04-18T22:22:33.127632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240784+0000","last_change":"2024-04-18T22:22:32.142740+0000","last_active":"2024-04-18T22:22:33.240784+0000","last_peered":"2024-04-18T22:22:33.240784+0000","last_clean":"2024-04-18T22:22:33.240784+0000","last_became_active":"2024-04-18T22:22:32.142048+0000","last_became_peered":"2024-04-18T22:22:32.142048+0000","last_unstale":"2024-04-18T22:22:33.240784+0000","last_undegraded":"2024-04-18T22:22:33.240784+0000","last_fullsized":"2024-04-18T22:22:33.240784+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240758+0000","last_change":"2024-04-18T22:22:32.129635+0000","last_active":"2024-04-18T22:22:33.240758+0000","last_peered":"2024-04-18T22:22:33.240758+0000","last_clean":"2024-04-18T22:22:33.240758+0000","last_became_active":"2024-04-18T22:22:32.128909+0000","last_became_peered":"2024-04-18T22:22:32.128909+0000","last_unstale":"2024-04-18T22:22:33.240758+0000","last_undegraded":"2024-04-18T22:22:33.240758+0000","last_fullsized":"2024-04-18T22:22:33.240758+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240016+0000","last_change":"2024-04-18T22:22:32.135755+0000","last_active":"2024-04-18T22:22:33.240016+0000","last_peered":"2024-04-18T22:22:33.240016+0000","last_clean":"2024-04-18T22:22:33.240016+0000","last_became_active":"2024-04-18T22:22:32.134588+0000","last_became_peered":"2024-04-18T22:22:32.134588+0000","last_unstale":"2024-04-18T22:22:33.240016+0000","last_undegraded":"2024-04-18T22:22:33.240016+0000","last_fullsized":"2024-04-18T22:22:33.240016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244495+0000","last_change":"2024-04-18T22:22:32.150194+0000","last_active":"2024-04-18T22:22:33.244495+0000","last_peered":"2024-04-18T22:22:33.244495+0000","last_clean":"2024-04-18T22:22:33.244495+0000","last_became_active":"2024-04-18T22:22:32.147919+0000","last_became_peered":"2024-04-18T22:22:32.147919+0000","last_unstale":"2024-04-18T22:22:33.244495+0000","last_undegraded":"2024-04-18T22:22:33.244495+0000","last_fullsized":"2024-04-18T22:22:33.244495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127840+0000","last_change":"2024-04-18T22:22:32.147917+0000","last_active":"2024-04-18T22:22:33.127840+0000","last_peered":"2024-04-18T22:22:33.127840+0000","last_clean":"2024-04-18T22:22:33.127840+0000","last_became_active":"2024-04-18T22:22:32.145953+0000","last_became_peered":"2024-04-18T22:22:32.145953+0000","last_unstale":"2024-04-18T22:22:33.127840+0000","last_undegraded":"2024-04-18T22:22:33.127840+0000","last_fullsized":"2024-04-18T22:22:33.127840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242825+0000","last_change":"2024-04-18T22:22:32.154655+0000","last_active":"2024-04-18T22:22:33.242825+0000","last_peered":"2024-04-18T22:22:33.242825+0000","last_clean":"2024-04-18T22:22:33.242825+0000","last_became_active":"2024-04-18T22:22:32.153966+0000","last_became_peered":"2024-04-18T22:22:32.153966+0000","last_unstale":"2024-04-18T22:22:33.242825+0000","last_undegraded":"2024-04-18T22:22:33.242825+0000","last_fullsized":"2024-04-18T22:22:33.242825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241750+0000","last_change":"2024-04-18T22:22:32.144928+0000","last_active":"2024-04-18T22:22:33.241750+0000","last_peered":"2024-04-18T22:22:33.241750+0000","last_clean":"2024-04-18T22:22:33.241750+0000","last_became_active":"2024-04-18T22:22:32.143282+0000","last_became_peered":"2024-04-18T22:22:32.143282+0000","last_unstale":"2024-04-18T22:22:33.241750+0000","last_undegraded":"2024-04-18T22:22:33.241750+0000","last_fullsized":"2024-04-18T22:22:33.241750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242442+0000","last_change":"2024-04-18T22:22:32.135811+0000","last_active":"2024-04-18T22:22:33.242442+0000","last_peered":"2024-04-18T22:22:33.242442+0000","last_clean":"2024-04-18T22:22:33.242442+0000","last_became_active":"2024-04-18T22:22:32.133269+0000","last_became_peered":"2024-04-18T22:22:32.133269+0000","last_unstale":"2024-04-18T22:22:33.242442+0000","last_undegraded":"2024-04-18T22:22:33.242442+0000","last_fullsized":"2024-04-18T22:22:33.242442+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:54:20.135916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126145+0000","last_change":"2024-04-18T22:22:32.141500+0000","last_active":"2024-04-18T22:22:33.126145+0000","last_peered":"2024-04-18T22:22:33.126145+0000","last_clean":"2024-04-18T22:22:33.126145+0000","last_became_active":"2024-04-18T22:22:32.140691+0000","last_became_peered":"2024-04-18T22:22:32.140691+0000","last_unstale":"2024-04-18T22:22:33.126145+0000","last_undegraded":"2024-04-18T22:22:33.126145+0000","last_fullsized":"2024-04-18T22:22:33.126145+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240965+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:22:33.240965+0000","last_peered":"2024-04-18T22:22:33.240965+0000","last_clean":"2024-04-18T22:22:33.240965+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:22:33.240965+0000","last_undegraded":"2024-04-18T22:22:33.240965+0000","last_fullsized":"2024-04-18T22:22:33.240965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126203+0000","last_change":"2024-04-18T22:22:32.141378+0000","last_active":"2024-04-18T22:22:33.126203+0000","last_peered":"2024-04-18T22:22:33.126203+0000","last_clean":"2024-04-18T22:22:33.126203+0000","last_became_active":"2024-04-18T22:22:32.140448+0000","last_became_peered":"2024-04-18T22:22:32.140448+0000","last_unstale":"2024-04-18T22:22:33.126203+0000","last_undegraded":"2024-04-18T22:22:33.126203+0000","last_fullsized":"2024-04-18T22:22:33.126203+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128370+0000","last_change":"2024-04-18T22:22:32.138865+0000","last_active":"2024-04-18T22:22:33.128370+0000","last_peered":"2024-04-18T22:22:33.128370+0000","last_clean":"2024-04-18T22:22:33.128370+0000","last_became_active":"2024-04-18T22:22:32.137793+0000","last_became_peered":"2024-04-18T22:22:32.137793+0000","last_unstale":"2024-04-18T22:22:33.128370+0000","last_undegraded":"2024-04-18T22:22:33.128370+0000","last_fullsized":"2024-04-18T22:22:33.128370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243541+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:22:33.243541+0000","last_peered":"2024-04-18T22:22:33.243541+0000","last_clean":"2024-04-18T22:22:33.243541+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:22:33.243541+0000","last_undegraded":"2024-04-18T22:22:33.243541+0000","last_fullsized":"2024-04-18T22:22:33.243541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242810+0000","last_change":"2024-04-18T22:22:32.146390+0000","last_active":"2024-04-18T22:22:33.242810+0000","last_peered":"2024-04-18T22:22:33.242810+0000","last_clean":"2024-04-18T22:22:33.242810+0000","last_became_active":"2024-04-18T22:22:32.145054+0000","last_became_peered":"2024-04-18T22:22:32.145054+0000","last_unstale":"2024-04-18T22:22:33.242810+0000","last_undegraded":"2024-04-18T22:22:33.242810+0000","last_fullsized":"2024-04-18T22:22:33.242810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:07:24.823488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127808+0000","last_change":"2024-04-18T22:22:32.132408+0000","last_active":"2024-04-18T22:22:33.127808+0000","last_peered":"2024-04-18T22:22:33.127808+0000","last_clean":"2024-04-18T22:22:33.127808+0000","last_became_active":"2024-04-18T22:22:32.131044+0000","last_became_peered":"2024-04-18T22:22:32.131044+0000","last_unstale":"2024-04-18T22:22:33.127808+0000","last_undegraded":"2024-04-18T22:22:33.127808+0000","last_fullsized":"2024-04-18T22:22:33.127808+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242233+0000","last_change":"2024-04-18T22:22:32.148879+0000","last_active":"2024-04-18T22:22:33.242233+0000","last_peered":"2024-04-18T22:22:33.242233+0000","last_clean":"2024-04-18T22:22:33.242233+0000","last_became_active":"2024-04-18T22:22:32.148070+0000","last_became_peered":"2024-04-18T22:22:32.148070+0000","last_unstale":"2024-04-18T22:22:33.242233+0000","last_undegraded":"2024-04-18T22:22:33.242233+0000","last_fullsized":"2024-04-18T22:22:33.242233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126582+0000","last_change":"2024-04-18T22:22:32.141405+0000","last_active":"2024-04-18T22:22:33.126582+0000","last_peered":"2024-04-18T22:22:33.126582+0000","last_clean":"2024-04-18T22:22:33.126582+0000","last_became_active":"2024-04-18T22:22:32.140369+0000","last_became_peered":"2024-04-18T22:22:32.140369+0000","last_unstale":"2024-04-18T22:22:33.126582+0000","last_undegraded":"2024-04-18T22:22:33.126582+0000","last_fullsized":"2024-04-18T22:22:33.126582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241205+0000","last_change":"2024-04-18T22:22:32.149208+0000","last_active":"2024-04-18T22:22:33.241205+0000","last_peered":"2024-04-18T22:22:33.241205+0000","last_clean":"2024-04-18T22:22:33.241205+0000","last_became_active":"2024-04-18T22:22:32.148344+0000","last_became_peered":"2024-04-18T22:22:32.148344+0000","last_unstale":"2024-04-18T22:22:33.241205+0000","last_undegraded":"2024-04-18T22:22:33.241205+0000","last_fullsized":"2024-04-18T22:22:33.241205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241714+0000","last_change":"2024-04-18T22:22:32.136695+0000","last_active":"2024-04-18T22:22:33.241714+0000","last_peered":"2024-04-18T22:22:33.241714+0000","last_clean":"2024-04-18T22:22:33.241714+0000","last_became_active":"2024-04-18T22:22:32.135901+0000","last_became_peered":"2024-04-18T22:22:32.135901+0000","last_unstale":"2024-04-18T22:22:33.241714+0000","last_undegraded":"2024-04-18T22:22:33.241714+0000","last_fullsized":"2024-04-18T22:22:33.241714+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127706+0000","last_change":"2024-04-18T22:22:32.144946+0000","last_active":"2024-04-18T22:22:33.127706+0000","last_peered":"2024-04-18T22:22:33.127706+0000","last_clean":"2024-04-18T22:22:33.127706+0000","last_became_active":"2024-04-18T22:22:32.143032+0000","last_became_peered":"2024-04-18T22:22:32.143032+0000","last_unstale":"2024-04-18T22:22:33.127706+0000","last_undegraded":"2024-04-18T22:22:33.127706+0000","last_fullsized":"2024-04-18T22:22:33.127706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242608+0000","last_change":"2024-04-18T22:22:31.145383+0000","last_active":"2024-04-18T22:22:33.242608+0000","last_peered":"2024-04-18T22:22:33.242608+0000","last_clean":"2024-04-18T22:22:33.242608+0000","last_became_active":"2024-04-18T22:22:31.143178+0000","last_became_peered":"2024-04-18T22:22:31.143178+0000","last_unstale":"2024-04-18T22:22:33.242608+0000","last_undegraded":"2024-04-18T22:22:33.242608+0000","last_fullsized":"2024-04-18T22:22:33.242608+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128208+0000","last_change":"2024-04-18T22:22:32.146608+0000","last_active":"2024-04-18T22:22:33.128208+0000","last_peered":"2024-04-18T22:22:33.128208+0000","last_clean":"2024-04-18T22:22:33.128208+0000","last_became_active":"2024-04-18T22:22:32.145761+0000","last_became_peered":"2024-04-18T22:22:32.145761+0000","last_unstale":"2024-04-18T22:22:33.128208+0000","last_undegraded":"2024-04-18T22:22:33.128208+0000","last_fullsized":"2024-04-18T22:22:33.128208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242107+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:22:33.242107+0000","last_peered":"2024-04-18T22:22:33.242107+0000","last_clean":"2024-04-18T22:22:33.242107+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:22:33.242107+0000","last_undegraded":"2024-04-18T22:22:33.242107+0000","last_fullsized":"2024-04-18T22:22:33.242107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243862+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:22:33.243862+0000","last_peered":"2024-04-18T22:22:33.243862+0000","last_clean":"2024-04-18T22:22:33.243862+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:22:33.243862+0000","last_undegraded":"2024-04-18T22:22:33.243862+0000","last_fullsized":"2024-04-18T22:22:33.243862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126186+0000","last_change":"2024-04-18T22:22:31.136552+0000","last_active":"2024-04-18T22:22:33.126186+0000","last_peered":"2024-04-18T22:22:33.126186+0000","last_clean":"2024-04-18T22:22:33.126186+0000","last_became_active":"2024-04-18T22:22:31.134824+0000","last_became_peered":"2024-04-18T22:22:31.134824+0000","last_unstale":"2024-04-18T22:22:33.126186+0000","last_undegraded":"2024-04-18T22:22:33.126186+0000","last_fullsized":"2024-04-18T22:22:33.126186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:07:17.380852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243033+0000","last_change":"2024-04-18T22:22:32.145785+0000","last_active":"2024-04-18T22:22:33.243033+0000","last_peered":"2024-04-18T22:22:33.243033+0000","last_clean":"2024-04-18T22:22:33.243033+0000","last_became_active":"2024-04-18T22:22:32.145166+0000","last_became_peered":"2024-04-18T22:22:32.145166+0000","last_unstale":"2024-04-18T22:22:33.243033+0000","last_undegraded":"2024-04-18T22:22:33.243033+0000","last_fullsized":"2024-04-18T22:22:33.243033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240502+0000","last_change":"2024-04-18T22:22:31.144229+0000","last_active":"2024-04-18T22:22:33.240502+0000","last_peered":"2024-04-18T22:22:33.240502+0000","last_clean":"2024-04-18T22:22:33.240502+0000","last_became_active":"2024-04-18T22:22:31.143597+0000","last_became_peered":"2024-04-18T22:22:31.143597+0000","last_unstale":"2024-04-18T22:22:33.240502+0000","last_undegraded":"2024-04-18T22:22:33.240502+0000","last_fullsized":"2024-04-18T22:22:33.240502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242009+0000","last_change":"2024-04-18T22:22:31.144343+0000","last_active":"2024-04-18T22:22:33.242009+0000","last_peered":"2024-04-18T22:22:33.242009+0000","last_clean":"2024-04-18T22:22:33.242009+0000","last_became_active":"2024-04-18T22:22:31.143325+0000","last_became_peered":"2024-04-18T22:22:31.143325+0000","last_unstale":"2024-04-18T22:22:33.242009+0000","last_undegraded":"2024-04-18T22:22:33.242009+0000","last_fullsized":"2024-04-18T22:22:33.242009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125867+0000","last_change":"2024-04-18T22:22:32.137726+0000","last_active":"2024-04-18T22:22:33.125867+0000","last_peered":"2024-04-18T22:22:33.125867+0000","last_clean":"2024-04-18T22:22:33.125867+0000","last_became_active":"2024-04-18T22:22:32.135216+0000","last_became_peered":"2024-04-18T22:22:32.135216+0000","last_unstale":"2024-04-18T22:22:33.125867+0000","last_undegraded":"2024-04-18T22:22:33.125867+0000","last_fullsized":"2024-04-18T22:22:33.125867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:01:43.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128180+0000","last_change":"2024-04-18T22:22:31.131784+0000","last_active":"2024-04-18T22:22:33.128180+0000","last_peered":"2024-04-18T22:22:33.128180+0000","last_clean":"2024-04-18T22:22:33.128180+0000","last_became_active":"2024-04-18T22:22:31.131002+0000","last_became_peered":"2024-04-18T22:22:31.131002+0000","last_unstale":"2024-04-18T22:22:33.128180+0000","last_undegraded":"2024-04-18T22:22:33.128180+0000","last_fullsized":"2024-04-18T22:22:33.128180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125897+0000","last_change":"2024-04-18T22:22:32.145000+0000","last_active":"2024-04-18T22:22:33.125897+0000","last_peered":"2024-04-18T22:22:33.125897+0000","last_clean":"2024-04-18T22:22:33.125897+0000","last_became_active":"2024-04-18T22:22:32.143522+0000","last_became_peered":"2024-04-18T22:22:32.143522+0000","last_unstale":"2024-04-18T22:22:33.125897+0000","last_undegraded":"2024-04-18T22:22:33.125897+0000","last_fullsized":"2024-04-18T22:22:33.125897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:26.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.144234+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.143532+0000","last_became_peered":"2024-04-18T22:22:31.143532+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:41.528679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241436+0000","last_change":"2024-04-18T22:22:32.135717+0000","last_active":"2024-04-18T22:22:33.241436+0000","last_peered":"2024-04-18T22:22:33.241436+0000","last_clean":"2024-04-18T22:22:33.241436+0000","last_became_active":"2024-04-18T22:22:32.134770+0000","last_became_peered":"2024-04-18T22:22:32.134770+0000","last_unstale":"2024-04-18T22:22:33.241436+0000","last_undegraded":"2024-04-18T22:22:33.241436+0000","last_fullsized":"2024-04-18T22:22:33.241436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242840+0000","last_change":"2024-04-18T22:22:31.147439+0000","last_active":"2024-04-18T22:22:33.242840+0000","last_peered":"2024-04-18T22:22:33.242840+0000","last_clean":"2024-04-18T22:22:33.242840+0000","last_became_active":"2024-04-18T22:22:31.146264+0000","last_became_peered":"2024-04-18T22:22:31.146264+0000","last_unstale":"2024-04-18T22:22:33.242840+0000","last_undegraded":"2024-04-18T22:22:33.242840+0000","last_fullsized":"2024-04-18T22:22:33.242840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242813+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:22:33.242813+0000","last_peered":"2024-04-18T22:22:33.242813+0000","last_clean":"2024-04-18T22:22:33.242813+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:22:33.242813+0000","last_undegraded":"2024-04-18T22:22:33.242813+0000","last_fullsized":"2024-04-18T22:22:33.242813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242737+0000","last_change":"2024-04-18T22:22:31.140451+0000","last_active":"2024-04-18T22:22:33.242737+0000","last_peered":"2024-04-18T22:22:33.242737+0000","last_clean":"2024-04-18T22:22:33.242737+0000","last_became_active":"2024-04-18T22:22:31.139655+0000","last_became_peered":"2024-04-18T22:22:31.139655+0000","last_unstale":"2024-04-18T22:22:33.242737+0000","last_undegraded":"2024-04-18T22:22:33.242737+0000","last_fullsized":"2024-04-18T22:22:33.242737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242161+0000","last_change":"2024-04-18T22:22:32.145869+0000","last_active":"2024-04-18T22:22:33.242161+0000","last_peered":"2024-04-18T22:22:33.242161+0000","last_clean":"2024-04-18T22:22:33.242161+0000","last_became_active":"2024-04-18T22:22:32.144117+0000","last_became_peered":"2024-04-18T22:22:32.144117+0000","last_unstale":"2024-04-18T22:22:33.242161+0000","last_undegraded":"2024-04-18T22:22:33.242161+0000","last_fullsized":"2024-04-18T22:22:33.242161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:39.186356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242361+0000","last_change":"2024-04-18T22:22:31.143197+0000","last_active":"2024-04-18T22:22:33.242361+0000","last_peered":"2024-04-18T22:22:33.242361+0000","last_clean":"2024-04-18T22:22:33.242361+0000","last_became_active":"2024-04-18T22:22:31.142393+0000","last_became_peered":"2024-04-18T22:22:31.142393+0000","last_unstale":"2024-04-18T22:22:33.242361+0000","last_undegraded":"2024-04-18T22:22:33.242361+0000","last_fullsized":"2024-04-18T22:22:33.242361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243118+0000","last_change":"2024-04-18T22:22:32.147396+0000","last_active":"2024-04-18T22:22:33.243118+0000","last_peered":"2024-04-18T22:22:33.243118+0000","last_clean":"2024-04-18T22:22:33.243118+0000","last_became_active":"2024-04-18T22:22:32.146327+0000","last_became_peered":"2024-04-18T22:22:32.146327+0000","last_unstale":"2024-04-18T22:22:33.243118+0000","last_undegraded":"2024-04-18T22:22:33.243118+0000","last_fullsized":"2024-04-18T22:22:33.243118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:16:31.884436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126713+0000","last_change":"2024-04-18T22:22:31.135058+0000","last_active":"2024-04-18T22:22:33.126713+0000","last_peered":"2024-04-18T22:22:33.126713+0000","last_clean":"2024-04-18T22:22:33.126713+0000","last_became_active":"2024-04-18T22:22:31.134213+0000","last_became_peered":"2024-04-18T22:22:31.134213+0000","last_unstale":"2024-04-18T22:22:33.126713+0000","last_undegraded":"2024-04-18T22:22:33.126713+0000","last_fullsized":"2024-04-18T22:22:33.126713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:35.837015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241581+0000","last_change":"2024-04-18T22:22:32.144956+0000","last_active":"2024-04-18T22:22:33.241581+0000","last_peered":"2024-04-18T22:22:33.241581+0000","last_clean":"2024-04-18T22:22:33.241581+0000","last_became_active":"2024-04-18T22:22:32.144171+0000","last_became_peered":"2024-04-18T22:22:32.144171+0000","last_unstale":"2024-04-18T22:22:33.241581+0000","last_undegraded":"2024-04-18T22:22:33.241581+0000","last_fullsized":"2024-04-18T22:22:33.241581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126257+0000","last_change":"2024-04-18T22:22:31.129222+0000","last_active":"2024-04-18T22:22:33.126257+0000","last_peered":"2024-04-18T22:22:33.126257+0000","last_clean":"2024-04-18T22:22:33.126257+0000","last_became_active":"2024-04-18T22:22:31.128618+0000","last_became_peered":"2024-04-18T22:22:31.128618+0000","last_unstale":"2024-04-18T22:22:33.126257+0000","last_undegraded":"2024-04-18T22:22:33.126257+0000","last_fullsized":"2024-04-18T22:22:33.126257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:58:49.687595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243356+0000","last_change":"2024-04-18T22:22:32.150911+0000","last_active":"2024-04-18T22:22:33.243356+0000","last_peered":"2024-04-18T22:22:33.243356+0000","last_clean":"2024-04-18T22:22:33.243356+0000","last_became_active":"2024-04-18T22:22:32.149188+0000","last_became_peered":"2024-04-18T22:22:32.149188+0000","last_unstale":"2024-04-18T22:22:33.243356+0000","last_undegraded":"2024-04-18T22:22:33.243356+0000","last_fullsized":"2024-04-18T22:22:33.243356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":222,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242605+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:33.242605+0000","last_peered":"2024-04-18T22:22:33.242605+0000","last_clean":"2024-04-18T22:22:33.242605+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:33.242605+0000","last_undegraded":"2024-04-18T22:22:33.242605+0000","last_fullsized":"2024-04-18T22:22:33.242605+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242816+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:33.242816+0000","last_peered":"2024-04-18T22:22:33.242816+0000","last_clean":"2024-04-18T22:22:33.242816+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:33.242816+0000","last_undegraded":"2024-04-18T22:22:33.242816+0000","last_fullsized":"2024-04-18T22:22:33.242816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242531+0000","last_change":"2024-04-18T22:22:31.128030+0000","last_active":"2024-04-18T22:22:33.242531+0000","last_peered":"2024-04-18T22:22:33.242531+0000","last_clean":"2024-04-18T22:22:33.242531+0000","last_became_active":"2024-04-18T22:22:31.126939+0000","last_became_peered":"2024-04-18T22:22:31.126939+0000","last_unstale":"2024-04-18T22:22:33.242531+0000","last_undegraded":"2024-04-18T22:22:33.242531+0000","last_fullsized":"2024-04-18T22:22:33.242531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243540+0000","last_change":"2024-04-18T22:22:32.153417+0000","last_active":"2024-04-18T22:22:33.243540+0000","last_peered":"2024-04-18T22:22:33.243540+0000","last_clean":"2024-04-18T22:22:33.243540+0000","last_became_active":"2024-04-18T22:22:32.152653+0000","last_became_peered":"2024-04-18T22:22:32.152653+0000","last_unstale":"2024-04-18T22:22:33.243540+0000","last_undegraded":"2024-04-18T22:22:33.243540+0000","last_fullsized":"2024-04-18T22:22:33.243540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:49:12.269993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242307+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:33.242307+0000","last_peered":"2024-04-18T22:22:33.242307+0000","last_clean":"2024-04-18T22:22:33.242307+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:33.242307+0000","last_undegraded":"2024-04-18T22:22:33.242307+0000","last_fullsized":"2024-04-18T22:22:33.242307+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242019+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:22:33.242019+0000","last_peered":"2024-04-18T22:22:33.242019+0000","last_clean":"2024-04-18T22:22:33.242019+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:22:33.242019+0000","last_undegraded":"2024-04-18T22:22:33.242019+0000","last_fullsized":"2024-04-18T22:22:33.242019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127898+0000","last_change":"2024-04-18T22:22:32.147651+0000","last_active":"2024-04-18T22:22:33.127898+0000","last_peered":"2024-04-18T22:22:33.127898+0000","last_clean":"2024-04-18T22:22:33.127898+0000","last_became_active":"2024-04-18T22:22:32.143774+0000","last_became_peered":"2024-04-18T22:22:32.143774+0000","last_unstale":"2024-04-18T22:22:33.127898+0000","last_undegraded":"2024-04-18T22:22:33.127898+0000","last_fullsized":"2024-04-18T22:22:33.127898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242167+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:33.242167+0000","last_peered":"2024-04-18T22:22:33.242167+0000","last_clean":"2024-04-18T22:22:33.242167+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:33.242167+0000","last_undegraded":"2024-04-18T22:22:33.242167+0000","last_fullsized":"2024-04-18T22:22:33.242167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126710+0000","last_change":"2024-04-18T22:22:31.132958+0000","last_active":"2024-04-18T22:22:33.126710+0000","last_peered":"2024-04-18T22:22:33.126710+0000","last_clean":"2024-04-18T22:22:33.126710+0000","last_became_active":"2024-04-18T22:22:31.132186+0000","last_became_peered":"2024-04-18T22:22:31.132186+0000","last_unstale":"2024-04-18T22:22:33.126710+0000","last_undegraded":"2024-04-18T22:22:33.126710+0000","last_fullsized":"2024-04-18T22:22:33.126710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243413+0000","last_change":"2024-04-18T22:22:32.150208+0000","last_active":"2024-04-18T22:22:33.243413+0000","last_peered":"2024-04-18T22:22:33.243413+0000","last_clean":"2024-04-18T22:22:33.243413+0000","last_became_active":"2024-04-18T22:22:32.148155+0000","last_became_peered":"2024-04-18T22:22:32.148155+0000","last_unstale":"2024-04-18T22:22:33.243413+0000","last_undegraded":"2024-04-18T22:22:33.243413+0000","last_fullsized":"2024-04-18T22:22:33.243413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241760+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:33.241760+0000","last_peered":"2024-04-18T22:22:33.241760+0000","last_clean":"2024-04-18T22:22:33.241760+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:33.241760+0000","last_undegraded":"2024-04-18T22:22:33.241760+0000","last_fullsized":"2024-04-18T22:22:33.241760+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242877+0000","last_change":"2024-04-18T22:22:31.146898+0000","last_active":"2024-04-18T22:22:33.242877+0000","last_peered":"2024-04-18T22:22:33.242877+0000","last_clean":"2024-04-18T22:22:33.242877+0000","last_became_active":"2024-04-18T22:22:31.146071+0000","last_became_peered":"2024-04-18T22:22:31.146071+0000","last_unstale":"2024-04-18T22:22:33.242877+0000","last_undegraded":"2024-04-18T22:22:33.242877+0000","last_fullsized":"2024-04-18T22:22:33.242877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243542+0000","last_change":"2024-04-18T22:22:32.147890+0000","last_active":"2024-04-18T22:22:33.243542+0000","last_peered":"2024-04-18T22:22:33.243542+0000","last_clean":"2024-04-18T22:22:33.243542+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.243542+0000","last_undegraded":"2024-04-18T22:22:33.243542+0000","last_fullsized":"2024-04-18T22:22:33.243542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243666+0000","last_change":"2024-04-18T22:22:31.146681+0000","last_active":"2024-04-18T22:22:33.243666+0000","last_peered":"2024-04-18T22:22:33.243666+0000","last_clean":"2024-04-18T22:22:33.243666+0000","last_became_active":"2024-04-18T22:22:31.145764+0000","last_became_peered":"2024-04-18T22:22:31.145764+0000","last_unstale":"2024-04-18T22:22:33.243666+0000","last_undegraded":"2024-04-18T22:22:33.243666+0000","last_fullsized":"2024-04-18T22:22:33.243666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:33.679846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243510+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:33.243510+0000","last_peered":"2024-04-18T22:22:33.243510+0000","last_clean":"2024-04-18T22:22:33.243510+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:33.243510+0000","last_undegraded":"2024-04-18T22:22:33.243510+0000","last_fullsized":"2024-04-18T22:22:33.243510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127522+0000","last_change":"2024-04-18T22:22:32.127971+0000","last_active":"2024-04-18T22:22:33.127522+0000","last_peered":"2024-04-18T22:22:33.127522+0000","last_clean":"2024-04-18T22:22:33.127522+0000","last_became_active":"2024-04-18T22:22:32.126518+0000","last_became_peered":"2024-04-18T22:22:32.126518+0000","last_unstale":"2024-04-18T22:22:33.127522+0000","last_undegraded":"2024-04-18T22:22:33.127522+0000","last_fullsized":"2024-04-18T22:22:33.127522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243415+0000","last_change":"2024-04-18T22:22:31.127787+0000","last_active":"2024-04-18T22:22:33.243415+0000","last_peered":"2024-04-18T22:22:33.243415+0000","last_clean":"2024-04-18T22:22:33.243415+0000","last_became_active":"2024-04-18T22:22:31.127173+0000","last_became_peered":"2024-04-18T22:22:31.127173+0000","last_unstale":"2024-04-18T22:22:33.243415+0000","last_undegraded":"2024-04-18T22:22:33.243415+0000","last_fullsized":"2024-04-18T22:22:33.243415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128505+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:33.128505+0000","last_peered":"2024-04-18T22:22:33.128505+0000","last_clean":"2024-04-18T22:22:33.128505+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:33.128505+0000","last_undegraded":"2024-04-18T22:22:33.128505+0000","last_fullsized":"2024-04-18T22:22:33.128505+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126910+0000","last_change":"2024-04-18T22:22:32.136161+0000","last_active":"2024-04-18T22:22:33.126910+0000","last_peered":"2024-04-18T22:22:33.126910+0000","last_clean":"2024-04-18T22:22:33.126910+0000","last_became_active":"2024-04-18T22:22:32.135253+0000","last_became_peered":"2024-04-18T22:22:32.135253+0000","last_unstale":"2024-04-18T22:22:33.126910+0000","last_undegraded":"2024-04-18T22:22:33.126910+0000","last_fullsized":"2024-04-18T22:22:33.126910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244171+0000","last_change":"2024-04-18T22:22:31.136319+0000","last_active":"2024-04-18T22:22:33.244171+0000","last_peered":"2024-04-18T22:22:33.244171+0000","last_clean":"2024-04-18T22:22:33.244171+0000","last_became_active":"2024-04-18T22:22:31.135689+0000","last_became_peered":"2024-04-18T22:22:31.135689+0000","last_unstale":"2024-04-18T22:22:33.244171+0000","last_undegraded":"2024-04-18T22:22:33.244171+0000","last_fullsized":"2024-04-18T22:22:33.244171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244003+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:33.244003+0000","last_peered":"2024-04-18T22:22:33.244003+0000","last_clean":"2024-04-18T22:22:33.244003+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:33.244003+0000","last_undegraded":"2024-04-18T22:22:33.244003+0000","last_fullsized":"2024-04-18T22:22:33.244003+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242936+0000","last_change":"2024-04-18T22:22:32.144432+0000","last_active":"2024-04-18T22:22:33.242936+0000","last_peered":"2024-04-18T22:22:33.242936+0000","last_clean":"2024-04-18T22:22:33.242936+0000","last_became_active":"2024-04-18T22:22:32.142947+0000","last_became_peered":"2024-04-18T22:22:32.142947+0000","last_unstale":"2024-04-18T22:22:33.242936+0000","last_undegraded":"2024-04-18T22:22:33.242936+0000","last_fullsized":"2024-04-18T22:22:33.242936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:34.119061+0000","last_change":"2024-04-18T22:22:31.143202+0000","last_active":"2024-04-18T22:22:34.119061+0000","last_peered":"2024-04-18T22:22:34.119061+0000","last_clean":"2024-04-18T22:22:34.119061+0000","last_became_active":"2024-04-18T22:22:31.142465+0000","last_became_peered":"2024-04-18T22:22:31.142465+0000","last_unstale":"2024-04-18T22:22:34.119061+0000","last_undegraded":"2024-04-18T22:22:34.119061+0000","last_fullsized":"2024-04-18T22:22:34.119061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241961+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:33.241961+0000","last_peered":"2024-04-18T22:22:33.241961+0000","last_clean":"2024-04-18T22:22:33.241961+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:33.241961+0000","last_undegraded":"2024-04-18T22:22:33.241961+0000","last_fullsized":"2024-04-18T22:22:33.241961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125912+0000","last_change":"2024-04-18T22:22:32.132968+0000","last_active":"2024-04-18T22:22:33.125912+0000","last_peered":"2024-04-18T22:22:33.125912+0000","last_clean":"2024-04-18T22:22:33.125912+0000","last_became_active":"2024-04-18T22:22:32.131596+0000","last_became_peered":"2024-04-18T22:22:32.131596+0000","last_unstale":"2024-04-18T22:22:33.125912+0000","last_undegraded":"2024-04-18T22:22:33.125912+0000","last_fullsized":"2024-04-18T22:22:33.125912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241097+0000","last_change":"2024-04-18T22:22:32.145633+0000","last_active":"2024-04-18T22:22:33.241097+0000","last_peered":"2024-04-18T22:22:33.241097+0000","last_clean":"2024-04-18T22:22:33.241097+0000","last_became_active":"2024-04-18T22:22:32.142868+0000","last_became_peered":"2024-04-18T22:22:32.142868+0000","last_unstale":"2024-04-18T22:22:33.241097+0000","last_undegraded":"2024-04-18T22:22:33.241097+0000","last_fullsized":"2024-04-18T22:22:33.241097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242663+0000","last_change":"2024-04-18T22:22:31.144413+0000","last_active":"2024-04-18T22:22:33.242663+0000","last_peered":"2024-04-18T22:22:33.242663+0000","last_clean":"2024-04-18T22:22:33.242663+0000","last_became_active":"2024-04-18T22:22:31.143704+0000","last_became_peered":"2024-04-18T22:22:31.143704+0000","last_unstale":"2024-04-18T22:22:33.242663+0000","last_undegraded":"2024-04-18T22:22:33.242663+0000","last_fullsized":"2024-04-18T22:22:33.242663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241992+0000","last_change":"2024-04-18T22:22:32.148407+0000","last_active":"2024-04-18T22:22:33.241992+0000","last_peered":"2024-04-18T22:22:33.241992+0000","last_clean":"2024-04-18T22:22:33.241992+0000","last_became_active":"2024-04-18T22:22:32.147033+0000","last_became_peered":"2024-04-18T22:22:32.147033+0000","last_unstale":"2024-04-18T22:22:33.241992+0000","last_undegraded":"2024-04-18T22:22:33.241992+0000","last_fullsized":"2024-04-18T22:22:33.241992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128409+0000","last_change":"2024-04-18T22:22:31.133306+0000","last_active":"2024-04-18T22:22:33.128409+0000","last_peered":"2024-04-18T22:22:33.128409+0000","last_clean":"2024-04-18T22:22:33.128409+0000","last_became_active":"2024-04-18T22:22:31.132648+0000","last_became_peered":"2024-04-18T22:22:31.132648+0000","last_unstale":"2024-04-18T22:22:33.128409+0000","last_undegraded":"2024-04-18T22:22:33.128409+0000","last_fullsized":"2024-04-18T22:22:33.128409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242551+0000","last_change":"2024-04-18T22:22:31.146827+0000","last_active":"2024-04-18T22:22:33.242551+0000","last_peered":"2024-04-18T22:22:33.242551+0000","last_clean":"2024-04-18T22:22:33.242551+0000","last_became_active":"2024-04-18T22:22:31.146063+0000","last_became_peered":"2024-04-18T22:22:31.146063+0000","last_unstale":"2024-04-18T22:22:33.242551+0000","last_undegraded":"2024-04-18T22:22:33.242551+0000","last_fullsized":"2024-04-18T22:22:33.242551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241955+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:22:33.241955+0000","last_peered":"2024-04-18T22:22:33.241955+0000","last_clean":"2024-04-18T22:22:33.241955+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:22:33.241955+0000","last_undegraded":"2024-04-18T22:22:33.241955+0000","last_fullsized":"2024-04-18T22:22:33.241955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243856+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:22:33.243856+0000","last_peered":"2024-04-18T22:22:33.243856+0000","last_clean":"2024-04-18T22:22:33.243856+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:22:33.243856+0000","last_undegraded":"2024-04-18T22:22:33.243856+0000","last_fullsized":"2024-04-18T22:22:33.243856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241179+0000","last_change":"2024-04-18T22:22:32.146927+0000","last_active":"2024-04-18T22:22:33.241179+0000","last_peered":"2024-04-18T22:22:33.241179+0000","last_clean":"2024-04-18T22:22:33.241179+0000","last_became_active":"2024-04-18T22:22:32.146040+0000","last_became_peered":"2024-04-18T22:22:32.146040+0000","last_unstale":"2024-04-18T22:22:33.241179+0000","last_undegraded":"2024-04-18T22:22:33.241179+0000","last_fullsized":"2024-04-18T22:22:33.241179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128951+0000","last_change":"2024-04-18T22:22:32.139063+0000","last_active":"2024-04-18T22:22:33.128951+0000","last_peered":"2024-04-18T22:22:33.128951+0000","last_clean":"2024-04-18T22:22:33.128951+0000","last_became_active":"2024-04-18T22:22:32.138016+0000","last_became_peered":"2024-04-18T22:22:32.138016+0000","last_unstale":"2024-04-18T22:22:33.128951+0000","last_undegraded":"2024-04-18T22:22:33.128951+0000","last_fullsized":"2024-04-18T22:22:33.128951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241615+0000","last_change":"2024-04-18T22:22:31.130452+0000","last_active":"2024-04-18T22:22:33.241615+0000","last_peered":"2024-04-18T22:22:33.241615+0000","last_clean":"2024-04-18T22:22:33.241615+0000","last_became_active":"2024-04-18T22:22:31.129555+0000","last_became_peered":"2024-04-18T22:22:31.129555+0000","last_unstale":"2024-04-18T22:22:33.241615+0000","last_undegraded":"2024-04-18T22:22:33.241615+0000","last_fullsized":"2024-04-18T22:22:33.241615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243327+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:22:33.243327+0000","last_peered":"2024-04-18T22:22:33.243327+0000","last_clean":"2024-04-18T22:22:33.243327+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:22:33.243327+0000","last_undegraded":"2024-04-18T22:22:33.243327+0000","last_fullsized":"2024-04-18T22:22:33.243327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241696+0000","last_change":"2024-04-18T22:22:31.133021+0000","last_active":"2024-04-18T22:22:33.241696+0000","last_peered":"2024-04-18T22:22:33.241696+0000","last_clean":"2024-04-18T22:22:33.241696+0000","last_became_active":"2024-04-18T22:22:31.132544+0000","last_became_peered":"2024-04-18T22:22:31.132544+0000","last_unstale":"2024-04-18T22:22:33.241696+0000","last_undegraded":"2024-04-18T22:22:33.241696+0000","last_fullsized":"2024-04-18T22:22:33.241696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127774+0000","last_change":"2024-04-18T22:22:32.138696+0000","last_active":"2024-04-18T22:22:33.127774+0000","last_peered":"2024-04-18T22:22:33.127774+0000","last_clean":"2024-04-18T22:22:33.127774+0000","last_became_active":"2024-04-18T22:22:32.137900+0000","last_became_peered":"2024-04-18T22:22:32.137900+0000","last_unstale":"2024-04-18T22:22:33.127774+0000","last_undegraded":"2024-04-18T22:22:33.127774+0000","last_fullsized":"2024-04-18T22:22:33.127774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126866+0000","last_change":"2024-04-18T22:22:31.133358+0000","last_active":"2024-04-18T22:22:33.126866+0000","last_peered":"2024-04-18T22:22:33.126866+0000","last_clean":"2024-04-18T22:22:33.126866+0000","last_became_active":"2024-04-18T22:22:31.132642+0000","last_became_peered":"2024-04-18T22:22:31.132642+0000","last_unstale":"2024-04-18T22:22:33.126866+0000","last_undegraded":"2024-04-18T22:22:33.126866+0000","last_fullsized":"2024-04-18T22:22:33.126866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127236+0000","last_change":"2024-04-18T22:22:32.146126+0000","last_active":"2024-04-18T22:22:33.127236+0000","last_peered":"2024-04-18T22:22:33.127236+0000","last_clean":"2024-04-18T22:22:33.127236+0000","last_became_active":"2024-04-18T22:22:32.144083+0000","last_became_peered":"2024-04-18T22:22:32.144083+0000","last_unstale":"2024-04-18T22:22:33.127236+0000","last_undegraded":"2024-04-18T22:22:33.127236+0000","last_fullsized":"2024-04-18T22:22:33.127236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127648+0000","last_change":"2024-04-18T22:22:31.136636+0000","last_active":"2024-04-18T22:22:33.127648+0000","last_peered":"2024-04-18T22:22:33.127648+0000","last_clean":"2024-04-18T22:22:33.127648+0000","last_became_active":"2024-04-18T22:22:31.135039+0000","last_became_peered":"2024-04-18T22:22:31.135039+0000","last_unstale":"2024-04-18T22:22:33.127648+0000","last_undegraded":"2024-04-18T22:22:33.127648+0000","last_fullsized":"2024-04-18T22:22:33.127648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.293851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240793+0000","last_change":"2024-04-18T22:22:32.134794+0000","last_active":"2024-04-18T22:22:33.240793+0000","last_peered":"2024-04-18T22:22:33.240793+0000","last_clean":"2024-04-18T22:22:33.240793+0000","last_became_active":"2024-04-18T22:22:32.133858+0000","last_became_peered":"2024-04-18T22:22:32.133858+0000","last_unstale":"2024-04-18T22:22:33.240793+0000","last_undegraded":"2024-04-18T22:22:33.240793+0000","last_fullsized":"2024-04-18T22:22:33.240793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242444+0000","last_change":"2024-04-18T22:22:31.147456+0000","last_active":"2024-04-18T22:22:33.242444+0000","last_peered":"2024-04-18T22:22:33.242444+0000","last_clean":"2024-04-18T22:22:33.242444+0000","last_became_active":"2024-04-18T22:22:31.145624+0000","last_became_peered":"2024-04-18T22:22:31.145624+0000","last_unstale":"2024-04-18T22:22:33.242444+0000","last_undegraded":"2024-04-18T22:22:33.242444+0000","last_fullsized":"2024-04-18T22:22:33.242444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241579+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.241579+0000","last_peered":"2024-04-18T22:22:33.241579+0000","last_clean":"2024-04-18T22:22:33.241579+0000","last_became_active":"2024-04-18T22:22:32.143184+0000","last_became_peered":"2024-04-18T22:22:32.143184+0000","last_unstale":"2024-04-18T22:22:33.241579+0000","last_undegraded":"2024-04-18T22:22:33.241579+0000","last_fullsized":"2024-04-18T22:22:33.241579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128174+0000","last_change":"2024-04-18T22:22:31.138492+0000","last_active":"2024-04-18T22:22:33.128174+0000","last_peered":"2024-04-18T22:22:33.128174+0000","last_clean":"2024-04-18T22:22:33.128174+0000","last_became_active":"2024-04-18T22:22:31.134646+0000","last_became_peered":"2024-04-18T22:22:31.134646+0000","last_unstale":"2024-04-18T22:22:33.128174+0000","last_undegraded":"2024-04-18T22:22:33.128174+0000","last_fullsized":"2024-04-18T22:22:33.128174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127146+0000","last_change":"2024-04-18T22:22:32.137473+0000","last_active":"2024-04-18T22:22:33.127146+0000","last_peered":"2024-04-18T22:22:33.127146+0000","last_clean":"2024-04-18T22:22:33.127146+0000","last_became_active":"2024-04-18T22:22:32.136025+0000","last_became_peered":"2024-04-18T22:22:32.136025+0000","last_unstale":"2024-04-18T22:22:33.127146+0000","last_undegraded":"2024-04-18T22:22:33.127146+0000","last_fullsized":"2024-04-18T22:22:33.127146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243824+0000","last_change":"2024-04-18T22:22:31.130827+0000","last_active":"2024-04-18T22:22:33.243824+0000","last_peered":"2024-04-18T22:22:33.243824+0000","last_clean":"2024-04-18T22:22:33.243824+0000","last_became_active":"2024-04-18T22:22:31.129728+0000","last_became_peered":"2024-04-18T22:22:31.129728+0000","last_unstale":"2024-04-18T22:22:33.243824+0000","last_undegraded":"2024-04-18T22:22:33.243824+0000","last_fullsized":"2024-04-18T22:22:33.243824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242152+0000","last_change":"2024-04-18T22:22:32.144325+0000","last_active":"2024-04-18T22:22:33.242152+0000","last_peered":"2024-04-18T22:22:33.242152+0000","last_clean":"2024-04-18T22:22:33.242152+0000","last_became_active":"2024-04-18T22:22:32.143495+0000","last_became_peered":"2024-04-18T22:22:32.143495+0000","last_unstale":"2024-04-18T22:22:33.242152+0000","last_undegraded":"2024-04-18T22:22:33.242152+0000","last_fullsized":"2024-04-18T22:22:33.242152+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.142640+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.141819+0000","last_became_peered":"2024-04-18T22:22:31.141819+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738376,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":245760,"data_stored":647890,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":647274,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":55728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:44.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-18T22:22:42.831636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":90020,"kb_used_data":632,"kb_used_omap":4,"kb_used_meta":89339,"kb_avail":283025500,"statfs":{"total":289910292480,"available":289818112000,"internally_reserved":0,"allocated":647168,"data_stored":1350892,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":91483471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003165"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128432+0000","last_change":"2024-04-18T22:22:32.145997+0000","last_active":"2024-04-18T22:22:33.128432+0000","last_peered":"2024-04-18T22:22:33.128432+0000","last_clean":"2024-04-18T22:22:33.128432+0000","last_became_active":"2024-04-18T22:22:32.143852+0000","last_became_peered":"2024-04-18T22:22:32.143852+0000","last_unstale":"2024-04-18T22:22:33.128432+0000","last_undegraded":"2024-04-18T22:22:33.128432+0000","last_fullsized":"2024-04-18T22:22:33.128432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126921+0000","last_change":"2024-04-18T22:22:32.148788+0000","last_active":"2024-04-18T22:22:33.126921+0000","last_peered":"2024-04-18T22:22:33.126921+0000","last_clean":"2024-04-18T22:22:33.126921+0000","last_became_active":"2024-04-18T22:22:32.146781+0000","last_became_peered":"2024-04-18T22:22:32.146781+0000","last_unstale":"2024-04-18T22:22:33.126921+0000","last_undegraded":"2024-04-18T22:22:33.126921+0000","last_fullsized":"2024-04-18T22:22:33.126921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241643+0000","last_change":"2024-04-18T22:22:32.152874+0000","last_active":"2024-04-18T22:22:33.241643+0000","last_peered":"2024-04-18T22:22:33.241643+0000","last_clean":"2024-04-18T22:22:33.241643+0000","last_became_active":"2024-04-18T22:22:32.152154+0000","last_became_peered":"2024-04-18T22:22:32.152154+0000","last_unstale":"2024-04-18T22:22:33.241643+0000","last_undegraded":"2024-04-18T22:22:33.241643+0000","last_fullsized":"2024-04-18T22:22:33.241643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242573+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.242573+0000","last_peered":"2024-04-18T22:22:33.242573+0000","last_clean":"2024-04-18T22:22:33.242573+0000","last_became_active":"2024-04-18T22:22:32.142591+0000","last_became_peered":"2024-04-18T22:22:32.142591+0000","last_unstale":"2024-04-18T22:22:33.242573+0000","last_undegraded":"2024-04-18T22:22:33.242573+0000","last_fullsized":"2024-04-18T22:22:33.242573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:24:32.079309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127358+0000","last_change":"2024-04-18T22:22:32.136433+0000","last_active":"2024-04-18T22:22:33.127358+0000","last_peered":"2024-04-18T22:22:33.127358+0000","last_clean":"2024-04-18T22:22:33.127358+0000","last_became_active":"2024-04-18T22:22:32.135586+0000","last_became_peered":"2024-04-18T22:22:32.135586+0000","last_unstale":"2024-04-18T22:22:33.127358+0000","last_undegraded":"2024-04-18T22:22:33.127358+0000","last_fullsized":"2024-04-18T22:22:33.127358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240606+0000","last_change":"2024-04-18T22:22:32.134957+0000","last_active":"2024-04-18T22:22:33.240606+0000","last_peered":"2024-04-18T22:22:33.240606+0000","last_clean":"2024-04-18T22:22:33.240606+0000","last_became_active":"2024-04-18T22:22:32.133797+0000","last_became_peered":"2024-04-18T22:22:32.133797+0000","last_unstale":"2024-04-18T22:22:33.240606+0000","last_undegraded":"2024-04-18T22:22:33.240606+0000","last_fullsized":"2024-04-18T22:22:33.240606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240547+0000","last_change":"2024-04-18T22:22:32.133775+0000","last_active":"2024-04-18T22:22:33.240547+0000","last_peered":"2024-04-18T22:22:33.240547+0000","last_clean":"2024-04-18T22:22:33.240547+0000","last_became_active":"2024-04-18T22:22:32.132972+0000","last_became_peered":"2024-04-18T22:22:32.132972+0000","last_unstale":"2024-04-18T22:22:33.240547+0000","last_undegraded":"2024-04-18T22:22:33.240547+0000","last_fullsized":"2024-04-18T22:22:33.240547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.239986+0000","last_change":"2024-04-18T22:22:32.136411+0000","last_active":"2024-04-18T22:22:33.239986+0000","last_peered":"2024-04-18T22:22:33.239986+0000","last_clean":"2024-04-18T22:22:33.239986+0000","last_became_active":"2024-04-18T22:22:32.132809+0000","last_became_peered":"2024-04-18T22:22:32.132809+0000","last_unstale":"2024-04-18T22:22:33.239986+0000","last_undegraded":"2024-04-18T22:22:33.239986+0000","last_fullsized":"2024-04-18T22:22:33.239986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242905+0000","last_change":"2024-04-18T22:22:32.150575+0000","last_active":"2024-04-18T22:22:33.242905+0000","last_peered":"2024-04-18T22:22:33.242905+0000","last_clean":"2024-04-18T22:22:33.242905+0000","last_became_active":"2024-04-18T22:22:32.149253+0000","last_became_peered":"2024-04-18T22:22:32.149253+0000","last_unstale":"2024-04-18T22:22:33.242905+0000","last_undegraded":"2024-04-18T22:22:33.242905+0000","last_fullsized":"2024-04-18T22:22:33.242905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127402+0000","last_change":"2024-04-18T22:22:32.148298+0000","last_active":"2024-04-18T22:22:33.127402+0000","last_peered":"2024-04-18T22:22:33.127402+0000","last_clean":"2024-04-18T22:22:33.127402+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.127402+0000","last_undegraded":"2024-04-18T22:22:33.127402+0000","last_fullsized":"2024-04-18T22:22:33.127402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242310+0000","last_change":"2024-04-18T22:22:32.133832+0000","last_active":"2024-04-18T22:22:33.242310+0000","last_peered":"2024-04-18T22:22:33.242310+0000","last_clean":"2024-04-18T22:22:33.242310+0000","last_became_active":"2024-04-18T22:22:32.132844+0000","last_became_peered":"2024-04-18T22:22:32.132844+0000","last_unstale":"2024-04-18T22:22:33.242310+0000","last_undegraded":"2024-04-18T22:22:33.242310+0000","last_fullsized":"2024-04-18T22:22:33.242310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:04:32.092674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126733+0000","last_change":"2024-04-18T22:22:32.140365+0000","last_active":"2024-04-18T22:22:33.126733+0000","last_peered":"2024-04-18T22:22:33.126733+0000","last_clean":"2024-04-18T22:22:33.126733+0000","last_became_active":"2024-04-18T22:22:32.138147+0000","last_became_peered":"2024-04-18T22:22:32.138147+0000","last_unstale":"2024-04-18T22:22:33.126733+0000","last_undegraded":"2024-04-18T22:22:33.126733+0000","last_fullsized":"2024-04-18T22:22:33.126733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127632+0000","last_change":"2024-04-18T22:22:32.146043+0000","last_active":"2024-04-18T22:22:33.127632+0000","last_peered":"2024-04-18T22:22:33.127632+0000","last_clean":"2024-04-18T22:22:33.127632+0000","last_became_active":"2024-04-18T22:22:32.144174+0000","last_became_peered":"2024-04-18T22:22:32.144174+0000","last_unstale":"2024-04-18T22:22:33.127632+0000","last_undegraded":"2024-04-18T22:22:33.127632+0000","last_fullsized":"2024-04-18T22:22:33.127632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240784+0000","last_change":"2024-04-18T22:22:32.142740+0000","last_active":"2024-04-18T22:22:33.240784+0000","last_peered":"2024-04-18T22:22:33.240784+0000","last_clean":"2024-04-18T22:22:33.240784+0000","last_became_active":"2024-04-18T22:22:32.142048+0000","last_became_peered":"2024-04-18T22:22:32.142048+0000","last_unstale":"2024-04-18T22:22:33.240784+0000","last_undegraded":"2024-04-18T22:22:33.240784+0000","last_fullsized":"2024-04-18T22:22:33.240784+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240758+0000","last_change":"2024-04-18T22:22:32.129635+0000","last_active":"2024-04-18T22:22:33.240758+0000","last_peered":"2024-04-18T22:22:33.240758+0000","last_clean":"2024-04-18T22:22:33.240758+0000","last_became_active":"2024-04-18T22:22:32.128909+0000","last_became_peered":"2024-04-18T22:22:32.128909+0000","last_unstale":"2024-04-18T22:22:33.240758+0000","last_undegraded":"2024-04-18T22:22:33.240758+0000","last_fullsized":"2024-04-18T22:22:33.240758+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240016+0000","last_change":"2024-04-18T22:22:32.135755+0000","last_active":"2024-04-18T22:22:33.240016+0000","last_peered":"2024-04-18T22:22:33.240016+0000","last_clean":"2024-04-18T22:22:33.240016+0000","last_became_active":"2024-04-18T22:22:32.134588+0000","last_became_peered":"2024-04-18T22:22:32.134588+0000","last_unstale":"2024-04-18T22:22:33.240016+0000","last_undegraded":"2024-04-18T22:22:33.240016+0000","last_fullsized":"2024-04-18T22:22:33.240016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244495+0000","last_change":"2024-04-18T22:22:32.150194+0000","last_active":"2024-04-18T22:22:33.244495+0000","last_peered":"2024-04-18T22:22:33.244495+0000","last_clean":"2024-04-18T22:22:33.244495+0000","last_became_active":"2024-04-18T22:22:32.147919+0000","last_became_peered":"2024-04-18T22:22:32.147919+0000","last_unstale":"2024-04-18T22:22:33.244495+0000","last_undegraded":"2024-04-18T22:22:33.244495+0000","last_fullsized":"2024-04-18T22:22:33.244495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127840+0000","last_change":"2024-04-18T22:22:32.147917+0000","last_active":"2024-04-18T22:22:33.127840+0000","last_peered":"2024-04-18T22:22:33.127840+0000","last_clean":"2024-04-18T22:22:33.127840+0000","last_became_active":"2024-04-18T22:22:32.145953+0000","last_became_peered":"2024-04-18T22:22:32.145953+0000","last_unstale":"2024-04-18T22:22:33.127840+0000","last_undegraded":"2024-04-18T22:22:33.127840+0000","last_fullsized":"2024-04-18T22:22:33.127840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242825+0000","last_change":"2024-04-18T22:22:32.154655+0000","last_active":"2024-04-18T22:22:33.242825+0000","last_peered":"2024-04-18T22:22:33.242825+0000","last_clean":"2024-04-18T22:22:33.242825+0000","last_became_active":"2024-04-18T22:22:32.153966+0000","last_became_peered":"2024-04-18T22:22:32.153966+0000","last_unstale":"2024-04-18T22:22:33.242825+0000","last_undegraded":"2024-04-18T22:22:33.242825+0000","last_fullsized":"2024-04-18T22:22:33.242825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241750+0000","last_change":"2024-04-18T22:22:32.144928+0000","last_active":"2024-04-18T22:22:33.241750+0000","last_peered":"2024-04-18T22:22:33.241750+0000","last_clean":"2024-04-18T22:22:33.241750+0000","last_became_active":"2024-04-18T22:22:32.143282+0000","last_became_peered":"2024-04-18T22:22:32.143282+0000","last_unstale":"2024-04-18T22:22:33.241750+0000","last_undegraded":"2024-04-18T22:22:33.241750+0000","last_fullsized":"2024-04-18T22:22:33.241750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242442+0000","last_change":"2024-04-18T22:22:32.135811+0000","last_active":"2024-04-18T22:22:33.242442+0000","last_peered":"2024-04-18T22:22:33.242442+0000","last_clean":"2024-04-18T22:22:33.242442+0000","last_became_active":"2024-04-18T22:22:32.133269+0000","last_became_peered":"2024-04-18T22:22:32.133269+0000","last_unstale":"2024-04-18T22:22:33.242442+0000","last_undegraded":"2024-04-18T22:22:33.242442+0000","last_fullsized":"2024-04-18T22:22:33.242442+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:54:20.135916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126145+0000","last_change":"2024-04-18T22:22:32.141500+0000","last_active":"2024-04-18T22:22:33.126145+0000","last_peered":"2024-04-18T22:22:33.126145+0000","last_clean":"2024-04-18T22:22:33.126145+0000","last_became_active":"2024-04-18T22:22:32.140691+0000","last_became_peered":"2024-04-18T22:22:32.140691+0000","last_unstale":"2024-04-18T22:22:33.126145+0000","last_undegraded":"2024-04-18T22:22:33.126145+0000","last_fullsized":"2024-04-18T22:22:33.126145+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240965+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:22:33.240965+0000","last_peered":"2024-04-18T22:22:33.240965+0000","last_clean":"2024-04-18T22:22:33.240965+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:22:33.240965+0000","last_undegraded":"2024-04-18T22:22:33.240965+0000","last_fullsized":"2024-04-18T22:22:33.240965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126203+0000","last_change":"2024-04-18T22:22:32.141378+0000","last_active":"2024-04-18T22:22:33.126203+0000","last_peered":"2024-04-18T22:22:33.126203+0000","last_clean":"2024-04-18T22:22:33.126203+0000","last_became_active":"2024-04-18T22:22:32.140448+0000","last_became_peered":"2024-04-18T22:22:32.140448+0000","last_unstale":"2024-04-18T22:22:33.126203+0000","last_undegraded":"2024-04-18T22:22:33.126203+0000","last_fullsized":"2024-04-18T22:22:33.126203+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128370+0000","last_change":"2024-04-18T22:22:32.138865+0000","last_active":"2024-04-18T22:22:33.128370+0000","last_peered":"2024-04-18T22:22:33.128370+0000","last_clean":"2024-04-18T22:22:33.128370+0000","last_became_active":"2024-04-18T22:22:32.137793+0000","last_became_peered":"2024-04-18T22:22:32.137793+0000","last_unstale":"2024-04-18T22:22:33.128370+0000","last_undegraded":"2024-04-18T22:22:33.128370+0000","last_fullsized":"2024-04-18T22:22:33.128370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243541+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:22:33.243541+0000","last_peered":"2024-04-18T22:22:33.243541+0000","last_clean":"2024-04-18T22:22:33.243541+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:22:33.243541+0000","last_undegraded":"2024-04-18T22:22:33.243541+0000","last_fullsized":"2024-04-18T22:22:33.243541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242810+0000","last_change":"2024-04-18T22:22:32.146390+0000","last_active":"2024-04-18T22:22:33.242810+0000","last_peered":"2024-04-18T22:22:33.242810+0000","last_clean":"2024-04-18T22:22:33.242810+0000","last_became_active":"2024-04-18T22:22:32.145054+0000","last_became_peered":"2024-04-18T22:22:32.145054+0000","last_unstale":"2024-04-18T22:22:33.242810+0000","last_undegraded":"2024-04-18T22:22:33.242810+0000","last_fullsized":"2024-04-18T22:22:33.242810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:07:24.823488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127808+0000","last_change":"2024-04-18T22:22:32.132408+0000","last_active":"2024-04-18T22:22:33.127808+0000","last_peered":"2024-04-18T22:22:33.127808+0000","last_clean":"2024-04-18T22:22:33.127808+0000","last_became_active":"2024-04-18T22:22:32.131044+0000","last_became_peered":"2024-04-18T22:22:32.131044+0000","last_unstale":"2024-04-18T22:22:33.127808+0000","last_undegraded":"2024-04-18T22:22:33.127808+0000","last_fullsized":"2024-04-18T22:22:33.127808+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242233+0000","last_change":"2024-04-18T22:22:32.148879+0000","last_active":"2024-04-18T22:22:33.242233+0000","last_peered":"2024-04-18T22:22:33.242233+0000","last_clean":"2024-04-18T22:22:33.242233+0000","last_became_active":"2024-04-18T22:22:32.148070+0000","last_became_peered":"2024-04-18T22:22:32.148070+0000","last_unstale":"2024-04-18T22:22:33.242233+0000","last_undegraded":"2024-04-18T22:22:33.242233+0000","last_fullsized":"2024-04-18T22:22:33.242233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126582+0000","last_change":"2024-04-18T22:22:32.141405+0000","last_active":"2024-04-18T22:22:33.126582+0000","last_peered":"2024-04-18T22:22:33.126582+0000","last_clean":"2024-04-18T22:22:33.126582+0000","last_became_active":"2024-04-18T22:22:32.140369+0000","last_became_peered":"2024-04-18T22:22:32.140369+0000","last_unstale":"2024-04-18T22:22:33.126582+0000","last_undegraded":"2024-04-18T22:22:33.126582+0000","last_fullsized":"2024-04-18T22:22:33.126582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241205+0000","last_change":"2024-04-18T22:22:32.149208+0000","last_active":"2024-04-18T22:22:33.241205+0000","last_peered":"2024-04-18T22:22:33.241205+0000","last_clean":"2024-04-18T22:22:33.241205+0000","last_became_active":"2024-04-18T22:22:32.148344+0000","last_became_peered":"2024-04-18T22:22:32.148344+0000","last_unstale":"2024-04-18T22:22:33.241205+0000","last_undegraded":"2024-04-18T22:22:33.241205+0000","last_fullsized":"2024-04-18T22:22:33.241205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241714+0000","last_change":"2024-04-18T22:22:32.136695+0000","last_active":"2024-04-18T22:22:33.241714+0000","last_peered":"2024-04-18T22:22:33.241714+0000","last_clean":"2024-04-18T22:22:33.241714+0000","last_became_active":"2024-04-18T22:22:32.135901+0000","last_became_peered":"2024-04-18T22:22:32.135901+0000","last_unstale":"2024-04-18T22:22:33.241714+0000","last_undegraded":"2024-04-18T22:22:33.241714+0000","last_fullsized":"2024-04-18T22:22:33.241714+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127706+0000","last_change":"2024-04-18T22:22:32.144946+0000","last_active":"2024-04-18T22:22:33.127706+0000","last_peered":"2024-04-18T22:22:33.127706+0000","last_clean":"2024-04-18T22:22:33.127706+0000","last_became_active":"2024-04-18T22:22:32.143032+0000","last_became_peered":"2024-04-18T22:22:32.143032+0000","last_unstale":"2024-04-18T22:22:33.127706+0000","last_undegraded":"2024-04-18T22:22:33.127706+0000","last_fullsized":"2024-04-18T22:22:33.127706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242608+0000","last_change":"2024-04-18T22:22:31.145383+0000","last_active":"2024-04-18T22:22:33.242608+0000","last_peered":"2024-04-18T22:22:33.242608+0000","last_clean":"2024-04-18T22:22:33.242608+0000","last_became_active":"2024-04-18T22:22:31.143178+0000","last_became_peered":"2024-04-18T22:22:31.143178+0000","last_unstale":"2024-04-18T22:22:33.242608+0000","last_undegraded":"2024-04-18T22:22:33.242608+0000","last_fullsized":"2024-04-18T22:22:33.242608+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128208+0000","last_change":"2024-04-18T22:22:32.146608+0000","last_active":"2024-04-18T22:22:33.128208+0000","last_peered":"2024-04-18T22:22:33.128208+0000","last_clean":"2024-04-18T22:22:33.128208+0000","last_became_active":"2024-04-18T22:22:32.145761+0000","last_became_peered":"2024-04-18T22:22:32.145761+0000","last_unstale":"2024-04-18T22:22:33.128208+0000","last_undegraded":"2024-04-18T22:22:33.128208+0000","last_fullsized":"2024-04-18T22:22:33.128208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242107+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:22:33.242107+0000","last_peered":"2024-04-18T22:22:33.242107+0000","last_clean":"2024-04-18T22:22:33.242107+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:22:33.242107+0000","last_undegraded":"2024-04-18T22:22:33.242107+0000","last_fullsized":"2024-04-18T22:22:33.242107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243862+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:22:33.243862+0000","last_peered":"2024-04-18T22:22:33.243862+0000","last_clean":"2024-04-18T22:22:33.243862+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:22:33.243862+0000","last_undegraded":"2024-04-18T22:22:33.243862+0000","last_fullsized":"2024-04-18T22:22:33.243862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126186+0000","last_change":"2024-04-18T22:22:31.136552+0000","last_active":"2024-04-18T22:22:33.126186+0000","last_peered":"2024-04-18T22:22:33.126186+0000","last_clean":"2024-04-18T22:22:33.126186+0000","last_became_active":"2024-04-18T22:22:31.134824+0000","last_became_peered":"2024-04-18T22:22:31.134824+0000","last_unstale":"2024-04-18T22:22:33.126186+0000","last_undegraded":"2024-04-18T22:22:33.126186+0000","last_fullsized":"2024-04-18T22:22:33.126186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:07:17.380852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243033+0000","last_change":"2024-04-18T22:22:32.145785+0000","last_active":"2024-04-18T22:22:33.243033+0000","last_peered":"2024-04-18T22:22:33.243033+0000","last_clean":"2024-04-18T22:22:33.243033+0000","last_became_active":"2024-04-18T22:22:32.145166+0000","last_became_peered":"2024-04-18T22:22:32.145166+0000","last_unstale":"2024-04-18T22:22:33.243033+0000","last_undegraded":"2024-04-18T22:22:33.243033+0000","last_fullsized":"2024-04-18T22:22:33.243033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240502+0000","last_change":"2024-04-18T22:22:31.144229+0000","last_active":"2024-04-18T22:22:33.240502+0000","last_peered":"2024-04-18T22:22:33.240502+0000","last_clean":"2024-04-18T22:22:33.240502+0000","last_became_active":"2024-04-18T22:22:31.143597+0000","last_became_peered":"2024-04-18T22:22:31.143597+0000","last_unstale":"2024-04-18T22:22:33.240502+0000","last_undegraded":"2024-04-18T22:22:33.240502+0000","last_fullsized":"2024-04-18T22:22:33.240502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242009+0000","last_change":"2024-04-18T22:22:31.144343+0000","last_active":"2024-04-18T22:22:33.242009+0000","last_peered":"2024-04-18T22:22:33.242009+0000","last_clean":"2024-04-18T22:22:33.242009+0000","last_became_active":"2024-04-18T22:22:31.143325+0000","last_became_peered":"2024-04-18T22:22:31.143325+0000","last_unstale":"2024-04-18T22:22:33.242009+0000","last_undegraded":"2024-04-18T22:22:33.242009+0000","last_fullsized":"2024-04-18T22:22:33.242009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125867+0000","last_change":"2024-04-18T22:22:32.137726+0000","last_active":"2024-04-18T22:22:33.125867+0000","last_peered":"2024-04-18T22:22:33.125867+0000","last_clean":"2024-04-18T22:22:33.125867+0000","last_became_active":"2024-04-18T22:22:32.135216+0000","last_became_peered":"2024-04-18T22:22:32.135216+0000","last_unstale":"2024-04-18T22:22:33.125867+0000","last_undegraded":"2024-04-18T22:22:33.125867+0000","last_fullsized":"2024-04-18T22:22:33.125867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:01:43.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128180+0000","last_change":"2024-04-18T22:22:31.131784+0000","last_active":"2024-04-18T22:22:33.128180+0000","last_peered":"2024-04-18T22:22:33.128180+0000","last_clean":"2024-04-18T22:22:33.128180+0000","last_became_active":"2024-04-18T22:22:31.131002+0000","last_became_peered":"2024-04-18T22:22:31.131002+0000","last_unstale":"2024-04-18T22:22:33.128180+0000","last_undegraded":"2024-04-18T22:22:33.128180+0000","last_fullsized":"2024-04-18T22:22:33.128180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125897+0000","last_change":"2024-04-18T22:22:32.145000+0000","last_active":"2024-04-18T22:22:33.125897+0000","last_peered":"2024-04-18T22:22:33.125897+0000","last_clean":"2024-04-18T22:22:33.125897+0000","last_became_active":"2024-04-18T22:22:32.143522+0000","last_became_peered":"2024-04-18T22:22:32.143522+0000","last_unstale":"2024-04-18T22:22:33.125897+0000","last_undegraded":"2024-04-18T22:22:33.125897+0000","last_fullsized":"2024-04-18T22:22:33.125897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:26.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.144234+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.143532+0000","last_became_peered":"2024-04-18T22:22:31.143532+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:41.528679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241436+0000","last_change":"2024-04-18T22:22:32.135717+0000","last_active":"2024-04-18T22:22:33.241436+0000","last_peered":"2024-04-18T22:22:33.241436+0000","last_clean":"2024-04-18T22:22:33.241436+0000","last_became_active":"2024-04-18T22:22:32.134770+0000","last_became_peered":"2024-04-18T22:22:32.134770+0000","last_unstale":"2024-04-18T22:22:33.241436+0000","last_undegraded":"2024-04-18T22:22:33.241436+0000","last_fullsized":"2024-04-18T22:22:33.241436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242840+0000","last_change":"2024-04-18T22:22:31.147439+0000","last_active":"2024-04-18T22:22:33.242840+0000","last_peered":"2024-04-18T22:22:33.242840+0000","last_clean":"2024-04-18T22:22:33.242840+0000","last_became_active":"2024-04-18T22:22:31.146264+0000","last_became_peered":"2024-04-18T22:22:31.146264+0000","last_unstale":"2024-04-18T22:22:33.242840+0000","last_undegraded":"2024-04-18T22:22:33.242840+0000","last_fullsized":"2024-04-18T22:22:33.242840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242813+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:22:33.242813+0000","last_peered":"2024-04-18T22:22:33.242813+0000","last_clean":"2024-04-18T22:22:33.242813+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:22:33.242813+0000","last_undegraded":"2024-04-18T22:22:33.242813+0000","last_fullsized":"2024-04-18T22:22:33.242813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242737+0000","last_change":"2024-04-18T22:22:31.140451+0000","last_active":"2024-04-18T22:22:33.242737+0000","last_peered":"2024-04-18T22:22:33.242737+0000","last_clean":"2024-04-18T22:22:33.242737+0000","last_became_active":"2024-04-18T22:22:31.139655+0000","last_became_peered":"2024-04-18T22:22:31.139655+0000","last_unstale":"2024-04-18T22:22:33.242737+0000","last_undegraded":"2024-04-18T22:22:33.242737+0000","last_fullsized":"2024-04-18T22:22:33.242737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242161+0000","last_change":"2024-04-18T22:22:32.145869+0000","last_active":"2024-04-18T22:22:33.242161+0000","last_peered":"2024-04-18T22:22:33.242161+0000","last_clean":"2024-04-18T22:22:33.242161+0000","last_became_active":"2024-04-18T22:22:32.144117+0000","last_became_peered":"2024-04-18T22:22:32.144117+0000","last_unstale":"2024-04-18T22:22:33.242161+0000","last_undegraded":"2024-04-18T22:22:33.242161+0000","last_fullsized":"2024-04-18T22:22:33.242161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:39.186356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242361+0000","last_change":"2024-04-18T22:22:31.143197+0000","last_active":"2024-04-18T22:22:33.242361+0000","last_peered":"2024-04-18T22:22:33.242361+0000","last_clean":"2024-04-18T22:22:33.242361+0000","last_became_active":"2024-04-18T22:22:31.142393+0000","last_became_peered":"2024-04-18T22:22:31.142393+0000","last_unstale":"2024-04-18T22:22:33.242361+0000","last_undegraded":"2024-04-18T22:22:33.242361+0000","last_fullsized":"2024-04-18T22:22:33.242361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243118+0000","last_change":"2024-04-18T22:22:32.147396+0000","last_active":"2024-04-18T22:22:33.243118+0000","last_peered":"2024-04-18T22:22:33.243118+0000","last_clean":"2024-04-18T22:22:33.243118+0000","last_became_active":"2024-04-18T22:22:32.146327+0000","last_became_peered":"2024-04-18T22:22:32.146327+0000","last_unstale":"2024-04-18T22:22:33.243118+0000","last_undegraded":"2024-04-18T22:22:33.243118+0000","last_fullsized":"2024-04-18T22:22:33.243118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:16:31.884436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126713+0000","last_change":"2024-04-18T22:22:31.135058+0000","last_active":"2024-04-18T22:22:33.126713+0000","last_peered":"2024-04-18T22:22:33.126713+0000","last_clean":"2024-04-18T22:22:33.126713+0000","last_became_active":"2024-04-18T22:22:31.134213+0000","last_became_peered":"2024-04-18T22:22:31.134213+0000","last_unstale":"2024-04-18T22:22:33.126713+0000","last_undegraded":"2024-04-18T22:22:33.126713+0000","last_fullsized":"2024-04-18T22:22:33.126713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:35.837015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241581+0000","last_change":"2024-04-18T22:22:32.144956+0000","last_active":"2024-04-18T22:22:33.241581+0000","last_peered":"2024-04-18T22:22:33.241581+0000","last_clean":"2024-04-18T22:22:33.241581+0000","last_became_active":"2024-04-18T22:22:32.144171+0000","last_became_peered":"2024-04-18T22:22:32.144171+0000","last_unstale":"2024-04-18T22:22:33.241581+0000","last_undegraded":"2024-04-18T22:22:33.241581+0000","last_fullsized":"2024-04-18T22:22:33.241581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126257+0000","last_change":"2024-04-18T22:22:31.129222+0000","last_active":"2024-04-18T22:22:33.126257+0000","last_peered":"2024-04-18T22:22:33.126257+0000","last_clean":"2024-04-18T22:22:33.126257+0000","last_became_active":"2024-04-18T22:22:31.128618+0000","last_became_peered":"2024-04-18T22:22:31.128618+0000","last_unstale":"2024-04-18T22:22:33.126257+0000","last_undegraded":"2024-04-18T22:22:33.126257+0000","last_fullsized":"2024-04-18T22:22:33.126257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:58:49.687595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243356+0000","last_change":"2024-04-18T22:22:32.150911+0000","last_active":"2024-04-18T22:22:33.243356+0000","last_peered":"2024-04-18T22:22:33.243356+0000","last_clean":"2024-04-18T22:22:33.243356+0000","last_became_active":"2024-04-18T22:22:32.149188+0000","last_became_peered":"2024-04-18T22:22:32.149188+0000","last_unstale":"2024-04-18T22:22:33.243356+0000","last_undegraded":"2024-04-18T22:22:33.243356+0000","last_fullsized":"2024-04-18T22:22:33.243356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":222,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242605+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:33.242605+0000","last_peered":"2024-04-18T22:22:33.242605+0000","last_clean":"2024-04-18T22:22:33.242605+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:33.242605+0000","last_undegraded":"2024-04-18T22:22:33.242605+0000","last_fullsized":"2024-04-18T22:22:33.242605+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242816+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:33.242816+0000","last_peered":"2024-04-18T22:22:33.242816+0000","last_clean":"2024-04-18T22:22:33.242816+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:33.242816+0000","last_undegraded":"2024-04-18T22:22:33.242816+0000","last_fullsized":"2024-04-18T22:22:33.242816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242531+0000","last_change":"2024-04-18T22:22:31.128030+0000","last_active":"2024-04-18T22:22:33.242531+0000","last_peered":"2024-04-18T22:22:33.242531+0000","last_clean":"2024-04-18T22:22:33.242531+0000","last_became_active":"2024-04-18T22:22:31.126939+0000","last_became_peered":"2024-04-18T22:22:31.126939+0000","last_unstale":"2024-04-18T22:22:33.242531+0000","last_undegraded":"2024-04-18T22:22:33.242531+0000","last_fullsized":"2024-04-18T22:22:33.242531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243540+0000","last_change":"2024-04-18T22:22:32.153417+0000","last_active":"2024-04-18T22:22:33.243540+0000","last_peered":"2024-04-18T22:22:33.243540+0000","last_clean":"2024-04-18T22:22:33.243540+0000","last_became_active":"2024-04-18T22:22:32.152653+0000","last_became_peered":"2024-04-18T22:22:32.152653+0000","last_unstale":"2024-04-18T22:22:33.243540+0000","last_undegraded":"2024-04-18T22:22:33.243540+0000","last_fullsized":"2024-04-18T22:22:33.243540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:49:12.269993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242307+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:33.242307+0000","last_peered":"2024-04-18T22:22:33.242307+0000","last_clean":"2024-04-18T22:22:33.242307+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:33.242307+0000","last_undegraded":"2024-04-18T22:22:33.242307+0000","last_fullsized":"2024-04-18T22:22:33.242307+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242019+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:22:33.242019+0000","last_peered":"2024-04-18T22:22:33.242019+0000","last_clean":"2024-04-18T22:22:33.242019+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:22:33.242019+0000","last_undegraded":"2024-04-18T22:22:33.242019+0000","last_fullsized":"2024-04-18T22:22:33.242019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127898+0000","last_change":"2024-04-18T22:22:32.147651+0000","last_active":"2024-04-18T22:22:33.127898+0000","last_peered":"2024-04-18T22:22:33.127898+0000","last_clean":"2024-04-18T22:22:33.127898+0000","last_became_active":"2024-04-18T22:22:32.143774+0000","last_became_peered":"2024-04-18T22:22:32.143774+0000","last_unstale":"2024-04-18T22:22:33.127898+0000","last_undegraded":"2024-04-18T22:22:33.127898+0000","last_fullsized":"2024-04-18T22:22:33.127898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242167+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:33.242167+0000","last_peered":"2024-04-18T22:22:33.242167+0000","last_clean":"2024-04-18T22:22:33.242167+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:33.242167+0000","last_undegraded":"2024-04-18T22:22:33.242167+0000","last_fullsized":"2024-04-18T22:22:33.242167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126710+0000","last_change":"2024-04-18T22:22:31.132958+0000","last_active":"2024-04-18T22:22:33.126710+0000","last_peered":"2024-04-18T22:22:33.126710+0000","last_clean":"2024-04-18T22:22:33.126710+0000","last_became_active":"2024-04-18T22:22:31.132186+0000","last_became_peered":"2024-04-18T22:22:31.132186+0000","last_unstale":"2024-04-18T22:22:33.126710+0000","last_undegraded":"2024-04-18T22:22:33.126710+0000","last_fullsized":"2024-04-18T22:22:33.126710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243413+0000","last_change":"2024-04-18T22:22:32.150208+0000","last_active":"2024-04-18T22:22:33.243413+0000","last_peered":"2024-04-18T22:22:33.243413+0000","last_clean":"2024-04-18T22:22:33.243413+0000","last_became_active":"2024-04-18T22:22:32.148155+0000","last_became_peered":"2024-04-18T22:22:32.148155+0000","last_unstale":"2024-04-18T22:22:33.243413+0000","last_undegraded":"2024-04-18T22:22:33.243413+0000","last_fullsized":"2024-04-18T22:22:33.243413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241760+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:33.241760+0000","last_peered":"2024-04-18T22:22:33.241760+0000","last_clean":"2024-04-18T22:22:33.241760+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:33.241760+0000","last_undegraded":"2024-04-18T22:22:33.241760+0000","last_fullsized":"2024-04-18T22:22:33.241760+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242877+0000","last_change":"2024-04-18T22:22:31.146898+0000","last_active":"2024-04-18T22:22:33.242877+0000","last_peered":"2024-04-18T22:22:33.242877+0000","last_clean":"2024-04-18T22:22:33.242877+0000","last_became_active":"2024-04-18T22:22:31.146071+0000","last_became_peered":"2024-04-18T22:22:31.146071+0000","last_unstale":"2024-04-18T22:22:33.242877+0000","last_undegraded":"2024-04-18T22:22:33.242877+0000","last_fullsized":"2024-04-18T22:22:33.242877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243542+0000","last_change":"2024-04-18T22:22:32.147890+0000","last_active":"2024-04-18T22:22:33.243542+0000","last_peered":"2024-04-18T22:22:33.243542+0000","last_clean":"2024-04-18T22:22:33.243542+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.243542+0000","last_undegraded":"2024-04-18T22:22:33.243542+0000","last_fullsized":"2024-04-18T22:22:33.243542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243666+0000","last_change":"2024-04-18T22:22:31.146681+0000","last_active":"2024-04-18T22:22:33.243666+0000","last_peered":"2024-04-18T22:22:33.243666+0000","last_clean":"2024-04-18T22:22:33.243666+0000","last_became_active":"2024-04-18T22:22:31.145764+0000","last_became_peered":"2024-04-18T22:22:31.145764+0000","last_unstale":"2024-04-18T22:22:33.243666+0000","last_undegraded":"2024-04-18T22:22:33.243666+0000","last_fullsized":"2024-04-18T22:22:33.243666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:33.679846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243510+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:33.243510+0000","last_peered":"2024-04-18T22:22:33.243510+0000","last_clean":"2024-04-18T22:22:33.243510+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:33.243510+0000","last_undegraded":"2024-04-18T22:22:33.243510+0000","last_fullsized":"2024-04-18T22:22:33.243510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127522+0000","last_change":"2024-04-18T22:22:32.127971+0000","last_active":"2024-04-18T22:22:33.127522+0000","last_peered":"2024-04-18T22:22:33.127522+0000","last_clean":"2024-04-18T22:22:33.127522+0000","last_became_active":"2024-04-18T22:22:32.126518+0000","last_became_peered":"2024-04-18T22:22:32.126518+0000","last_unstale":"2024-04-18T22:22:33.127522+0000","last_undegraded":"2024-04-18T22:22:33.127522+0000","last_fullsized":"2024-04-18T22:22:33.127522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243415+0000","last_change":"2024-04-18T22:22:31.127787+0000","last_active":"2024-04-18T22:22:33.243415+0000","last_peered":"2024-04-18T22:22:33.243415+0000","last_clean":"2024-04-18T22:22:33.243415+0000","last_became_active":"2024-04-18T22:22:31.127173+0000","last_became_peered":"2024-04-18T22:22:31.127173+0000","last_unstale":"2024-04-18T22:22:33.243415+0000","last_undegraded":"2024-04-18T22:22:33.243415+0000","last_fullsized":"2024-04-18T22:22:33.243415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128505+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:33.128505+0000","last_peered":"2024-04-18T22:22:33.128505+0000","last_clean":"2024-04-18T22:22:33.128505+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:33.128505+0000","last_undegraded":"2024-04-18T22:22:33.128505+0000","last_fullsized":"2024-04-18T22:22:33.128505+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126910+0000","last_change":"2024-04-18T22:22:32.136161+0000","last_active":"2024-04-18T22:22:33.126910+0000","last_peered":"2024-04-18T22:22:33.126910+0000","last_clean":"2024-04-18T22:22:33.126910+0000","last_became_active":"2024-04-18T22:22:32.135253+0000","last_became_peered":"2024-04-18T22:22:32.135253+0000","last_unstale":"2024-04-18T22:22:33.126910+0000","last_undegraded":"2024-04-18T22:22:33.126910+0000","last_fullsized":"2024-04-18T22:22:33.126910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244171+0000","last_change":"2024-04-18T22:22:31.136319+0000","last_active":"2024-04-18T22:22:33.244171+0000","last_peered":"2024-04-18T22:22:33.244171+0000","last_clean":"2024-04-18T22:22:33.244171+0000","last_became_active":"2024-04-18T22:22:31.135689+0000","last_became_peered":"2024-04-18T22:22:31.135689+0000","last_unstale":"2024-04-18T22:22:33.244171+0000","last_undegraded":"2024-04-18T22:22:33.244171+0000","last_fullsized":"2024-04-18T22:22:33.244171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244003+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:33.244003+0000","last_peered":"2024-04-18T22:22:33.244003+0000","last_clean":"2024-04-18T22:22:33.244003+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:33.244003+0000","last_undegraded":"2024-04-18T22:22:33.244003+0000","last_fullsized":"2024-04-18T22:22:33.244003+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242936+0000","last_change":"2024-04-18T22:22:32.144432+0000","last_active":"2024-04-18T22:22:33.242936+0000","last_peered":"2024-04-18T22:22:33.242936+0000","last_clean":"2024-04-18T22:22:33.242936+0000","last_became_active":"2024-04-18T22:22:32.142947+0000","last_became_peered":"2024-04-18T22:22:32.142947+0000","last_unstale":"2024-04-18T22:22:33.242936+0000","last_undegraded":"2024-04-18T22:22:33.242936+0000","last_fullsized":"2024-04-18T22:22:33.242936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:34.119061+0000","last_change":"2024-04-18T22:22:31.143202+0000","last_active":"2024-04-18T22:22:34.119061+0000","last_peered":"2024-04-18T22:22:34.119061+0000","last_clean":"2024-04-18T22:22:34.119061+0000","last_became_active":"2024-04-18T22:22:31.142465+0000","last_became_peered":"2024-04-18T22:22:31.142465+0000","last_unstale":"2024-04-18T22:22:34.119061+0000","last_undegraded":"2024-04-18T22:22:34.119061+0000","last_fullsized":"2024-04-18T22:22:34.119061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241961+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:33.241961+0000","last_peered":"2024-04-18T22:22:33.241961+0000","last_clean":"2024-04-18T22:22:33.241961+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:33.241961+0000","last_undegraded":"2024-04-18T22:22:33.241961+0000","last_fullsized":"2024-04-18T22:22:33.241961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125912+0000","last_change":"2024-04-18T22:22:32.132968+0000","last_active":"2024-04-18T22:22:33.125912+0000","last_peered":"2024-04-18T22:22:33.125912+0000","last_clean":"2024-04-18T22:22:33.125912+0000","last_became_active":"2024-04-18T22:22:32.131596+0000","last_became_peered":"2024-04-18T22:22:32.131596+0000","last_unstale":"2024-04-18T22:22:33.125912+0000","last_undegraded":"2024-04-18T22:22:33.125912+0000","last_fullsized":"2024-04-18T22:22:33.125912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241097+0000","last_change":"2024-04-18T22:22:32.145633+0000","last_active":"2024-04-18T22:22:33.241097+0000","last_peered":"2024-04-18T22:22:33.241097+0000","last_clean":"2024-04-18T22:22:33.241097+0000","last_became_active":"2024-04-18T22:22:32.142868+0000","last_became_peered":"2024-04-18T22:22:32.142868+0000","last_unstale":"2024-04-18T22:22:33.241097+0000","last_undegraded":"2024-04-18T22:22:33.241097+0000","last_fullsized":"2024-04-18T22:22:33.241097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242663+0000","last_change":"2024-04-18T22:22:31.144413+0000","last_active":"2024-04-18T22:22:33.242663+0000","last_peered":"2024-04-18T22:22:33.242663+0000","last_clean":"2024-04-18T22:22:33.242663+0000","last_became_active":"2024-04-18T22:22:31.143704+0000","last_became_peered":"2024-04-18T22:22:31.143704+0000","last_unstale":"2024-04-18T22:22:33.242663+0000","last_undegraded":"2024-04-18T22:22:33.242663+0000","last_fullsized":"2024-04-18T22:22:33.242663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241992+0000","last_change":"2024-04-18T22:22:32.148407+0000","last_active":"2024-04-18T22:22:33.241992+0000","last_peered":"2024-04-18T22:22:33.241992+0000","last_clean":"2024-04-18T22:22:33.241992+0000","last_became_active":"2024-04-18T22:22:32.147033+0000","last_became_peered":"2024-04-18T22:22:32.147033+0000","last_unstale":"2024-04-18T22:22:33.241992+0000","last_undegraded":"2024-04-18T22:22:33.241992+0000","last_fullsized":"2024-04-18T22:22:33.241992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128409+0000","last_change":"2024-04-18T22:22:31.133306+0000","last_active":"2024-04-18T22:22:33.128409+0000","last_peered":"2024-04-18T22:22:33.128409+0000","last_clean":"2024-04-18T22:22:33.128409+0000","last_became_active":"2024-04-18T22:22:31.132648+0000","last_became_peered":"2024-04-18T22:22:31.132648+0000","last_unstale":"2024-04-18T22:22:33.128409+0000","last_undegraded":"2024-04-18T22:22:33.128409+0000","last_fullsized":"2024-04-18T22:22:33.128409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242551+0000","last_change":"2024-04-18T22:22:31.146827+0000","last_active":"2024-04-18T22:22:33.242551+0000","last_peered":"2024-04-18T22:22:33.242551+0000","last_clean":"2024-04-18T22:22:33.242551+0000","last_became_active":"2024-04-18T22:22:31.146063+0000","last_became_peered":"2024-04-18T22:22:31.146063+0000","last_unstale":"2024-04-18T22:22:33.242551+0000","last_undegraded":"2024-04-18T22:22:33.242551+0000","last_fullsized":"2024-04-18T22:22:33.242551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241955+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:22:33.241955+0000","last_peered":"2024-04-18T22:22:33.241955+0000","last_clean":"2024-04-18T22:22:33.241955+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:22:33.241955+0000","last_undegraded":"2024-04-18T22:22:33.241955+0000","last_fullsized":"2024-04-18T22:22:33.241955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243856+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:22:33.243856+0000","last_peered":"2024-04-18T22:22:33.243856+0000","last_clean":"2024-04-18T22:22:33.243856+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:22:33.243856+0000","last_undegraded":"2024-04-18T22:22:33.243856+0000","last_fullsized":"2024-04-18T22:22:33.243856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241179+0000","last_change":"2024-04-18T22:22:32.146927+0000","last_active":"2024-04-18T22:22:33.241179+0000","last_peered":"2024-04-18T22:22:33.241179+0000","last_clean":"2024-04-18T22:22:33.241179+0000","last_became_active":"2024-04-18T22:22:32.146040+0000","last_became_peered":"2024-04-18T22:22:32.146040+0000","last_unstale":"2024-04-18T22:22:33.241179+0000","last_undegraded":"2024-04-18T22:22:33.241179+0000","last_fullsized":"2024-04-18T22:22:33.241179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128951+0000","last_change":"2024-04-18T22:22:32.139063+0000","last_active":"2024-04-18T22:22:33.128951+0000","last_peered":"2024-04-18T22:22:33.128951+0000","last_clean":"2024-04-18T22:22:33.128951+0000","last_became_active":"2024-04-18T22:22:32.138016+0000","last_became_peered":"2024-04-18T22:22:32.138016+0000","last_unstale":"2024-04-18T22:22:33.128951+0000","last_undegraded":"2024-04-18T22:22:33.128951+0000","last_fullsized":"2024-04-18T22:22:33.128951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241615+0000","last_change":"2024-04-18T22:22:31.130452+0000","last_active":"2024-04-18T22:22:33.241615+0000","last_peered":"2024-04-18T22:22:33.241615+0000","last_clean":"2024-04-18T22:22:33.241615+0000","last_became_active":"2024-04-18T22:22:31.129555+0000","last_became_peered":"2024-04-18T22:22:31.129555+0000","last_unstale":"2024-04-18T22:22:33.241615+0000","last_undegraded":"2024-04-18T22:22:33.241615+0000","last_fullsized":"2024-04-18T22:22:33.241615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243327+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:22:33.243327+0000","last_peered":"2024-04-18T22:22:33.243327+0000","last_clean":"2024-04-18T22:22:33.243327+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:22:33.243327+0000","last_undegraded":"2024-04-18T22:22:33.243327+0000","last_fullsized":"2024-04-18T22:22:33.243327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241696+0000","last_change":"2024-04-18T22:22:31.133021+0000","last_active":"2024-04-18T22:22:33.241696+0000","last_peered":"2024-04-18T22:22:33.241696+0000","last_clean":"2024-04-18T22:22:33.241696+0000","last_became_active":"2024-04-18T22:22:31.132544+0000","last_became_peered":"2024-04-18T22:22:31.132544+0000","last_unstale":"2024-04-18T22:22:33.241696+0000","last_undegraded":"2024-04-18T22:22:33.241696+0000","last_fullsized":"2024-04-18T22:22:33.241696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127774+0000","last_change":"2024-04-18T22:22:32.138696+0000","last_active":"2024-04-18T22:22:33.127774+0000","last_peered":"2024-04-18T22:22:33.127774+0000","last_clean":"2024-04-18T22:22:33.127774+0000","last_became_active":"2024-04-18T22:22:32.137900+0000","last_became_peered":"2024-04-18T22:22:32.137900+0000","last_unstale":"2024-04-18T22:22:33.127774+0000","last_undegraded":"2024-04-18T22:22:33.127774+0000","last_fullsized":"2024-04-18T22:22:33.127774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126866+0000","last_change":"2024-04-18T22:22:31.133358+0000","last_active":"2024-04-18T22:22:33.126866+0000","last_peered":"2024-04-18T22:22:33.126866+0000","last_clean":"2024-04-18T22:22:33.126866+0000","last_became_active":"2024-04-18T22:22:31.132642+0000","last_became_peered":"2024-04-18T22:22:31.132642+0000","last_unstale":"2024-04-18T22:22:33.126866+0000","last_undegraded":"2024-04-18T22:22:33.126866+0000","last_fullsized":"2024-04-18T22:22:33.126866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127236+0000","last_change":"2024-04-18T22:22:32.146126+0000","last_active":"2024-04-18T22:22:33.127236+0000","last_peered":"2024-04-18T22:22:33.127236+0000","last_clean":"2024-04-18T22:22:33.127236+0000","last_became_active":"2024-04-18T22:22:32.144083+0000","last_became_peered":"2024-04-18T22:22:32.144083+0000","last_unstale":"2024-04-18T22:22:33.127236+0000","last_undegraded":"2024-04-18T22:22:33.127236+0000","last_fullsized":"2024-04-18T22:22:33.127236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127648+0000","last_change":"2024-04-18T22:22:31.136636+0000","last_active":"2024-04-18T22:22:33.127648+0000","last_peered":"2024-04-18T22:22:33.127648+0000","last_clean":"2024-04-18T22:22:33.127648+0000","last_became_active":"2024-04-18T22:22:31.135039+0000","last_became_peered":"2024-04-18T22:22:31.135039+0000","last_unstale":"2024-04-18T22:22:33.127648+0000","last_undegraded":"2024-04-18T22:22:33.127648+0000","last_fullsized":"2024-04-18T22:22:33.127648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.293851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240793+0000","last_change":"2024-04-18T22:22:32.134794+0000","last_active":"2024-04-18T22:22:33.240793+0000","last_peered":"2024-04-18T22:22:33.240793+0000","last_clean":"2024-04-18T22:22:33.240793+0000","last_became_active":"2024-04-18T22:22:32.133858+0000","last_became_peered":"2024-04-18T22:22:32.133858+0000","last_unstale":"2024-04-18T22:22:33.240793+0000","last_undegraded":"2024-04-18T22:22:33.240793+0000","last_fullsized":"2024-04-18T22:22:33.240793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242444+0000","last_change":"2024-04-18T22:22:31.147456+0000","last_active":"2024-04-18T22:22:33.242444+0000","last_peered":"2024-04-18T22:22:33.242444+0000","last_clean":"2024-04-18T22:22:33.242444+0000","last_became_active":"2024-04-18T22:22:31.145624+0000","last_became_peered":"2024-04-18T22:22:31.145624+0000","last_unstale":"2024-04-18T22:22:33.242444+0000","last_undegraded":"2024-04-18T22:22:33.242444+0000","last_fullsized":"2024-04-18T22:22:33.242444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241579+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.241579+0000","last_peered":"2024-04-18T22:22:33.241579+0000","last_clean":"2024-04-18T22:22:33.241579+0000","last_became_active":"2024-04-18T22:22:32.143184+0000","last_became_peered":"2024-04-18T22:22:32.143184+0000","last_unstale":"2024-04-18T22:22:33.241579+0000","last_undegraded":"2024-04-18T22:22:33.241579+0000","last_fullsized":"2024-04-18T22:22:33.241579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128174+0000","last_change":"2024-04-18T22:22:31.138492+0000","last_active":"2024-04-18T22:22:33.128174+0000","last_peered":"2024-04-18T22:22:33.128174+0000","last_clean":"2024-04-18T22:22:33.128174+0000","last_became_active":"2024-04-18T22:22:31.134646+0000","last_became_peered":"2024-04-18T22:22:31.134646+0000","last_unstale":"2024-04-18T22:22:33.128174+0000","last_undegraded":"2024-04-18T22:22:33.128174+0000","last_fullsized":"2024-04-18T22:22:33.128174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127146+0000","last_change":"2024-04-18T22:22:32.137473+0000","last_active":"2024-04-18T22:22:33.127146+0000","last_peered":"2024-04-18T22:22:33.127146+0000","last_clean":"2024-04-18T22:22:33.127146+0000","last_became_active":"2024-04-18T22:22:32.136025+0000","last_became_peered":"2024-04-18T22:22:32.136025+0000","last_unstale":"2024-04-18T22:22:33.127146+0000","last_undegraded":"2024-04-18T22:22:33.127146+0000","last_fullsized":"2024-04-18T22:22:33.127146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243824+0000","last_change":"2024-04-18T22:22:31.130827+0000","last_active":"2024-04-18T22:22:33.243824+0000","last_peered":"2024-04-18T22:22:33.243824+0000","last_clean":"2024-04-18T22:22:33.243824+0000","last_became_active":"2024-04-18T22:22:31.129728+0000","last_became_peered":"2024-04-18T22:22:31.129728+0000","last_unstale":"2024-04-18T22:22:33.243824+0000","last_undegraded":"2024-04-18T22:22:33.243824+0000","last_fullsized":"2024-04-18T22:22:33.243824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242152+0000","last_change":"2024-04-18T22:22:32.144325+0000","last_active":"2024-04-18T22:22:33.242152+0000","last_peered":"2024-04-18T22:22:33.242152+0000","last_clean":"2024-04-18T22:22:33.242152+0000","last_became_active":"2024-04-18T22:22:32.143495+0000","last_became_peered":"2024-04-18T22:22:32.143495+0000","last_unstale":"2024-04-18T22:22:33.242152+0000","last_undegraded":"2024-04-18T22:22:33.242152+0000","last_fullsized":"2024-04-18T22:22:33.242152+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.142640+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.141819+0000","last_became_peered":"2024-04-18T22:22:31.141819+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738376,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":245760,"data_stored":647890,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":647274,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":55728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:44.847 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-18T22:22:44.847 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-18T22:22:44.848 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-18T22:22:44.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-18T22:22:45.209 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-18T22:22:45.226 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-18T22:22:45.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:45.514 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:45.515 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:33.115398+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:45.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-18T22:22:20.901399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-18T22:22:26.016101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-18T22:22:29.965363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-04-18T22:22:30.463150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-18T22:22:45.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-18T22:22:45.825 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-18T22:22:45.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-18T22:22:46.133 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2024-04-18T22:22:46.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-18T22:22:46.435 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-18T22:22:46.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-18T22:22:46.733 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-18T22:22:46.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:22:47.068 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:47.068 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-18T22:22:31.467948+0000","modified":"2024-04-18T22:22:33.473846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-04-18T22:22:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-18T22:22:47.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:47.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:47.364 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:33.115398+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:47.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-18T22:22:20.901399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-18T22:22:26.016101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-18T22:22:29.965363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-04-18T22:22:30.463150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-18T22:22:47.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-18T22:22:47.670 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-18T22:22:47.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-18T22:22:47.972 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2024-04-18T22:22:47.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-18T22:22:48.278 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-18T22:22:48.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-18T22:22:48.600 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-18T22:22:48.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:22:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:48.949 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-18T22:22:31.467948+0000","modified":"2024-04-18T22:22:33.473846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-04-18T22:22:48.949 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-18T22:22:48.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:22:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:49.263 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:22:33.115398+0000","last_up_change":"2024-04-18T22:22:19.704344+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6801","nonce":3490435725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6803","nonce":3490435725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6807","nonce":3490435725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3490435725},{"type":"v1","addr":"172.21.15.96:6805","nonce":3490435725}]},"public_addr":"172.21.15.96:6801/3490435725","cluster_addr":"172.21.15.96:6803/3490435725","heartbeat_back_addr":"172.21.15.96:6807/3490435725","heartbeat_front_addr":"172.21.15.96:6805/3490435725","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:22:49.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:22:49.609 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:49.609 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-18T22:22:31.467948+0000","modified":"2024-04-18T22:22:33.473846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-04-18T22:22:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-18T22:22:49.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-18T22:22:31.467948+0000', 'modified': '2024-04-18T22:22:33.473846+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4205': {'gid': 4205, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6803/2739131151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2739131151}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2739131151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4205, 'qdb_cluster': [4205]} 2024-04-18T22:22:49.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-18T22:22:49.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-18T22:22:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-18T22:22:49.871+0000 7f59bdffb640 0 client.4498 ms_handle_reset on v2:172.21.15.32:6802/2739131151 2024-04-18T22:22:49.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-18T22:22:49.895+0000 7f59bdffb640 0 client.4514 ms_handle_reset on v2:172.21.15.32:6802/2739131151 2024-04-18T22:22:49.894 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:22:49.894 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "a1b9e9bc-8f1f-4426-9796-fae392070eec", 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4205, 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 17.786575353, 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 5, 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 18, 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 18, 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 22.709832245000001 2024-04-18T22:22:49.895 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:22:49.910 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-18T22:22:49.921 INFO:tasks.thrashosds:config is {'chance_down': 1.0, 'powercycle': True, 'timeout': 600, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-18T22:22:49.921 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd0bf63dd179c05751e4be49477389aa290c73089'}} 2024-04-18T22:22:49.921 INFO:tasks.thrashosds:config is {'chance_down': 1.0, 'powercycle': True, 'timeout': 600, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-18T22:22:49.921 INFO:tasks.thrashosds:Doing preliminary sync to avoid collateral damage... 2024-04-18T22:22:49.921 DEBUG:teuthology.orchestra.run.smithi032:> sync 2024-04-18T22:22:50.050 DEBUG:teuthology.orchestra.run.smithi086:> sync 2024-04-18T22:22:50.215 DEBUG:teuthology.orchestra.run.smithi096:> sync 2024-04-18T22:22:50.421 DEBUG:teuthology.orchestra.run.smithi196:> sync 2024-04-18T22:22:50.508 DEBUG:tasks.thrashosds:checking console status of smithi032 2024-04-18T22:22:50.509 DEBUG:teuthology.orchestra.console.smithi032:Waiting for login prompt 2024-04-18T22:22:50.509 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi032 2024-04-18T22:22:50.615 DEBUG:teuthology.orchestra.console.smithi032:expect before: [Enter `^Ec?' for help] 2024-04-18T22:22:50.615 DEBUG:teuthology.orchestra.console.smithi032:expect after: smithi032 login: 2024-04-18T22:22:50.667 DEBUG:teuthology.orchestra.console.smithi032:console disconnect output: [Enter `^Ec?' for help] smithi032 login: [disconnect] 2024-04-18T22:22:50.769 DEBUG:tasks.thrashosds:checking console status of smithi086 2024-04-18T22:22:50.769 DEBUG:teuthology.orchestra.console.smithi086:Waiting for login prompt 2024-04-18T22:22:50.769 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi086 2024-04-18T22:22:50.855 DEBUG:teuthology.orchestra.console.smithi086:expect before: [Enter `^Ec?' for help] 2024-04-18T22:22:50.855 DEBUG:teuthology.orchestra.console.smithi086:expect after: smithi086 login: 2024-04-18T22:22:50.907 DEBUG:teuthology.orchestra.console.smithi086:console disconnect output: [Enter `^Ec?' for help] smithi086 login: [disconnect] 2024-04-18T22:22:51.008 DEBUG:tasks.thrashosds:checking console status of smithi096 2024-04-18T22:22:51.008 DEBUG:teuthology.orchestra.console.smithi096:Waiting for login prompt 2024-04-18T22:22:51.008 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi096 2024-04-18T22:22:51.098 DEBUG:teuthology.orchestra.console.smithi096:expect before: [Enter `^Ec?' for help] 2024-04-18T22:22:51.098 DEBUG:teuthology.orchestra.console.smithi096:expect after: smithi096 login: 2024-04-18T22:22:51.150 DEBUG:teuthology.orchestra.console.smithi096:console disconnect output: [Enter `^Ec?' for help] smithi096 login: [disconnect] 2024-04-18T22:22:51.251 DEBUG:tasks.thrashosds:checking console status of smithi196 2024-04-18T22:22:51.252 DEBUG:teuthology.orchestra.console.smithi196:Waiting for login prompt 2024-04-18T22:22:51.252 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi196 2024-04-18T22:22:51.413 DEBUG:teuthology.orchestra.console.smithi196:expect before: [Enter `^Ec?' for help] 2024-04-18T22:22:51.414 DEBUG:teuthology.orchestra.console.smithi196:expect after: smithi196 login: 2024-04-18T22:22:51.466 DEBUG:teuthology.orchestra.console.smithi196:console disconnect output: [Enter `^Ec?' for help] smithi196 login: [disconnect] 2024-04-18T22:22:51.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-18T22:22:51.905 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-18T22:22:51.905 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-18T22:22:51.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:52.232 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-18T22:22:50.833692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":90020,"kb_used_data":632,"kb_used_omap":4,"kb_used_meta":89339,"kb_avail":283025500,"statfs":{"total":289910292480,"available":289818112000,"internally_reserved":0,"allocated":647168,"data_stored":1350892,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":91483471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003563"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128432+0000","last_change":"2024-04-18T22:22:32.145997+0000","last_active":"2024-04-18T22:22:33.128432+0000","last_peered":"2024-04-18T22:22:33.128432+0000","last_clean":"2024-04-18T22:22:33.128432+0000","last_became_active":"2024-04-18T22:22:32.143852+0000","last_became_peered":"2024-04-18T22:22:32.143852+0000","last_unstale":"2024-04-18T22:22:33.128432+0000","last_undegraded":"2024-04-18T22:22:33.128432+0000","last_fullsized":"2024-04-18T22:22:33.128432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126921+0000","last_change":"2024-04-18T22:22:32.148788+0000","last_active":"2024-04-18T22:22:33.126921+0000","last_peered":"2024-04-18T22:22:33.126921+0000","last_clean":"2024-04-18T22:22:33.126921+0000","last_became_active":"2024-04-18T22:22:32.146781+0000","last_became_peered":"2024-04-18T22:22:32.146781+0000","last_unstale":"2024-04-18T22:22:33.126921+0000","last_undegraded":"2024-04-18T22:22:33.126921+0000","last_fullsized":"2024-04-18T22:22:33.126921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241643+0000","last_change":"2024-04-18T22:22:32.152874+0000","last_active":"2024-04-18T22:22:33.241643+0000","last_peered":"2024-04-18T22:22:33.241643+0000","last_clean":"2024-04-18T22:22:33.241643+0000","last_became_active":"2024-04-18T22:22:32.152154+0000","last_became_peered":"2024-04-18T22:22:32.152154+0000","last_unstale":"2024-04-18T22:22:33.241643+0000","last_undegraded":"2024-04-18T22:22:33.241643+0000","last_fullsized":"2024-04-18T22:22:33.241643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242573+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.242573+0000","last_peered":"2024-04-18T22:22:33.242573+0000","last_clean":"2024-04-18T22:22:33.242573+0000","last_became_active":"2024-04-18T22:22:32.142591+0000","last_became_peered":"2024-04-18T22:22:32.142591+0000","last_unstale":"2024-04-18T22:22:33.242573+0000","last_undegraded":"2024-04-18T22:22:33.242573+0000","last_fullsized":"2024-04-18T22:22:33.242573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:24:32.079309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127358+0000","last_change":"2024-04-18T22:22:32.136433+0000","last_active":"2024-04-18T22:22:33.127358+0000","last_peered":"2024-04-18T22:22:33.127358+0000","last_clean":"2024-04-18T22:22:33.127358+0000","last_became_active":"2024-04-18T22:22:32.135586+0000","last_became_peered":"2024-04-18T22:22:32.135586+0000","last_unstale":"2024-04-18T22:22:33.127358+0000","last_undegraded":"2024-04-18T22:22:33.127358+0000","last_fullsized":"2024-04-18T22:22:33.127358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240606+0000","last_change":"2024-04-18T22:22:32.134957+0000","last_active":"2024-04-18T22:22:33.240606+0000","last_peered":"2024-04-18T22:22:33.240606+0000","last_clean":"2024-04-18T22:22:33.240606+0000","last_became_active":"2024-04-18T22:22:32.133797+0000","last_became_peered":"2024-04-18T22:22:32.133797+0000","last_unstale":"2024-04-18T22:22:33.240606+0000","last_undegraded":"2024-04-18T22:22:33.240606+0000","last_fullsized":"2024-04-18T22:22:33.240606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240547+0000","last_change":"2024-04-18T22:22:32.133775+0000","last_active":"2024-04-18T22:22:33.240547+0000","last_peered":"2024-04-18T22:22:33.240547+0000","last_clean":"2024-04-18T22:22:33.240547+0000","last_became_active":"2024-04-18T22:22:32.132972+0000","last_became_peered":"2024-04-18T22:22:32.132972+0000","last_unstale":"2024-04-18T22:22:33.240547+0000","last_undegraded":"2024-04-18T22:22:33.240547+0000","last_fullsized":"2024-04-18T22:22:33.240547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.239986+0000","last_change":"2024-04-18T22:22:32.136411+0000","last_active":"2024-04-18T22:22:33.239986+0000","last_peered":"2024-04-18T22:22:33.239986+0000","last_clean":"2024-04-18T22:22:33.239986+0000","last_became_active":"2024-04-18T22:22:32.132809+0000","last_became_peered":"2024-04-18T22:22:32.132809+0000","last_unstale":"2024-04-18T22:22:33.239986+0000","last_undegraded":"2024-04-18T22:22:33.239986+0000","last_fullsized":"2024-04-18T22:22:33.239986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242905+0000","last_change":"2024-04-18T22:22:32.150575+0000","last_active":"2024-04-18T22:22:33.242905+0000","last_peered":"2024-04-18T22:22:33.242905+0000","last_clean":"2024-04-18T22:22:33.242905+0000","last_became_active":"2024-04-18T22:22:32.149253+0000","last_became_peered":"2024-04-18T22:22:32.149253+0000","last_unstale":"2024-04-18T22:22:33.242905+0000","last_undegraded":"2024-04-18T22:22:33.242905+0000","last_fullsized":"2024-04-18T22:22:33.242905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127402+0000","last_change":"2024-04-18T22:22:32.148298+0000","last_active":"2024-04-18T22:22:33.127402+0000","last_peered":"2024-04-18T22:22:33.127402+0000","last_clean":"2024-04-18T22:22:33.127402+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.127402+0000","last_undegraded":"2024-04-18T22:22:33.127402+0000","last_fullsized":"2024-04-18T22:22:33.127402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242310+0000","last_change":"2024-04-18T22:22:32.133832+0000","last_active":"2024-04-18T22:22:33.242310+0000","last_peered":"2024-04-18T22:22:33.242310+0000","last_clean":"2024-04-18T22:22:33.242310+0000","last_became_active":"2024-04-18T22:22:32.132844+0000","last_became_peered":"2024-04-18T22:22:32.132844+0000","last_unstale":"2024-04-18T22:22:33.242310+0000","last_undegraded":"2024-04-18T22:22:33.242310+0000","last_fullsized":"2024-04-18T22:22:33.242310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:04:32.092674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126733+0000","last_change":"2024-04-18T22:22:32.140365+0000","last_active":"2024-04-18T22:22:33.126733+0000","last_peered":"2024-04-18T22:22:33.126733+0000","last_clean":"2024-04-18T22:22:33.126733+0000","last_became_active":"2024-04-18T22:22:32.138147+0000","last_became_peered":"2024-04-18T22:22:32.138147+0000","last_unstale":"2024-04-18T22:22:33.126733+0000","last_undegraded":"2024-04-18T22:22:33.126733+0000","last_fullsized":"2024-04-18T22:22:33.126733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127632+0000","last_change":"2024-04-18T22:22:32.146043+0000","last_active":"2024-04-18T22:22:33.127632+0000","last_peered":"2024-04-18T22:22:33.127632+0000","last_clean":"2024-04-18T22:22:33.127632+0000","last_became_active":"2024-04-18T22:22:32.144174+0000","last_became_peered":"2024-04-18T22:22:32.144174+0000","last_unstale":"2024-04-18T22:22:33.127632+0000","last_undegraded":"2024-04-18T22:22:33.127632+0000","last_fullsized":"2024-04-18T22:22:33.127632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240784+0000","last_change":"2024-04-18T22:22:32.142740+0000","last_active":"2024-04-18T22:22:33.240784+0000","last_peered":"2024-04-18T22:22:33.240784+0000","last_clean":"2024-04-18T22:22:33.240784+0000","last_became_active":"2024-04-18T22:22:32.142048+0000","last_became_peered":"2024-04-18T22:22:32.142048+0000","last_unstale":"2024-04-18T22:22:33.240784+0000","last_undegraded":"2024-04-18T22:22:33.240784+0000","last_fullsized":"2024-04-18T22:22:33.240784+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240758+0000","last_change":"2024-04-18T22:22:32.129635+0000","last_active":"2024-04-18T22:22:33.240758+0000","last_peered":"2024-04-18T22:22:33.240758+0000","last_clean":"2024-04-18T22:22:33.240758+0000","last_became_active":"2024-04-18T22:22:32.128909+0000","last_became_peered":"2024-04-18T22:22:32.128909+0000","last_unstale":"2024-04-18T22:22:33.240758+0000","last_undegraded":"2024-04-18T22:22:33.240758+0000","last_fullsized":"2024-04-18T22:22:33.240758+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240016+0000","last_change":"2024-04-18T22:22:32.135755+0000","last_active":"2024-04-18T22:22:33.240016+0000","last_peered":"2024-04-18T22:22:33.240016+0000","last_clean":"2024-04-18T22:22:33.240016+0000","last_became_active":"2024-04-18T22:22:32.134588+0000","last_became_peered":"2024-04-18T22:22:32.134588+0000","last_unstale":"2024-04-18T22:22:33.240016+0000","last_undegraded":"2024-04-18T22:22:33.240016+0000","last_fullsized":"2024-04-18T22:22:33.240016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244495+0000","last_change":"2024-04-18T22:22:32.150194+0000","last_active":"2024-04-18T22:22:33.244495+0000","last_peered":"2024-04-18T22:22:33.244495+0000","last_clean":"2024-04-18T22:22:33.244495+0000","last_became_active":"2024-04-18T22:22:32.147919+0000","last_became_peered":"2024-04-18T22:22:32.147919+0000","last_unstale":"2024-04-18T22:22:33.244495+0000","last_undegraded":"2024-04-18T22:22:33.244495+0000","last_fullsized":"2024-04-18T22:22:33.244495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127840+0000","last_change":"2024-04-18T22:22:32.147917+0000","last_active":"2024-04-18T22:22:33.127840+0000","last_peered":"2024-04-18T22:22:33.127840+0000","last_clean":"2024-04-18T22:22:33.127840+0000","last_became_active":"2024-04-18T22:22:32.145953+0000","last_became_peered":"2024-04-18T22:22:32.145953+0000","last_unstale":"2024-04-18T22:22:33.127840+0000","last_undegraded":"2024-04-18T22:22:33.127840+0000","last_fullsized":"2024-04-18T22:22:33.127840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242825+0000","last_change":"2024-04-18T22:22:32.154655+0000","last_active":"2024-04-18T22:22:33.242825+0000","last_peered":"2024-04-18T22:22:33.242825+0000","last_clean":"2024-04-18T22:22:33.242825+0000","last_became_active":"2024-04-18T22:22:32.153966+0000","last_became_peered":"2024-04-18T22:22:32.153966+0000","last_unstale":"2024-04-18T22:22:33.242825+0000","last_undegraded":"2024-04-18T22:22:33.242825+0000","last_fullsized":"2024-04-18T22:22:33.242825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241750+0000","last_change":"2024-04-18T22:22:32.144928+0000","last_active":"2024-04-18T22:22:33.241750+0000","last_peered":"2024-04-18T22:22:33.241750+0000","last_clean":"2024-04-18T22:22:33.241750+0000","last_became_active":"2024-04-18T22:22:32.143282+0000","last_became_peered":"2024-04-18T22:22:32.143282+0000","last_unstale":"2024-04-18T22:22:33.241750+0000","last_undegraded":"2024-04-18T22:22:33.241750+0000","last_fullsized":"2024-04-18T22:22:33.241750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242442+0000","last_change":"2024-04-18T22:22:32.135811+0000","last_active":"2024-04-18T22:22:33.242442+0000","last_peered":"2024-04-18T22:22:33.242442+0000","last_clean":"2024-04-18T22:22:33.242442+0000","last_became_active":"2024-04-18T22:22:32.133269+0000","last_became_peered":"2024-04-18T22:22:32.133269+0000","last_unstale":"2024-04-18T22:22:33.242442+0000","last_undegraded":"2024-04-18T22:22:33.242442+0000","last_fullsized":"2024-04-18T22:22:33.242442+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:54:20.135916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126145+0000","last_change":"2024-04-18T22:22:32.141500+0000","last_active":"2024-04-18T22:22:33.126145+0000","last_peered":"2024-04-18T22:22:33.126145+0000","last_clean":"2024-04-18T22:22:33.126145+0000","last_became_active":"2024-04-18T22:22:32.140691+0000","last_became_peered":"2024-04-18T22:22:32.140691+0000","last_unstale":"2024-04-18T22:22:33.126145+0000","last_undegraded":"2024-04-18T22:22:33.126145+0000","last_fullsized":"2024-04-18T22:22:33.126145+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240965+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:22:33.240965+0000","last_peered":"2024-04-18T22:22:33.240965+0000","last_clean":"2024-04-18T22:22:33.240965+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:22:33.240965+0000","last_undegraded":"2024-04-18T22:22:33.240965+0000","last_fullsized":"2024-04-18T22:22:33.240965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126203+0000","last_change":"2024-04-18T22:22:32.141378+0000","last_active":"2024-04-18T22:22:33.126203+0000","last_peered":"2024-04-18T22:22:33.126203+0000","last_clean":"2024-04-18T22:22:33.126203+0000","last_became_active":"2024-04-18T22:22:32.140448+0000","last_became_peered":"2024-04-18T22:22:32.140448+0000","last_unstale":"2024-04-18T22:22:33.126203+0000","last_undegraded":"2024-04-18T22:22:33.126203+0000","last_fullsized":"2024-04-18T22:22:33.126203+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128370+0000","last_change":"2024-04-18T22:22:32.138865+0000","last_active":"2024-04-18T22:22:33.128370+0000","last_peered":"2024-04-18T22:22:33.128370+0000","last_clean":"2024-04-18T22:22:33.128370+0000","last_became_active":"2024-04-18T22:22:32.137793+0000","last_became_peered":"2024-04-18T22:22:32.137793+0000","last_unstale":"2024-04-18T22:22:33.128370+0000","last_undegraded":"2024-04-18T22:22:33.128370+0000","last_fullsized":"2024-04-18T22:22:33.128370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243541+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:22:33.243541+0000","last_peered":"2024-04-18T22:22:33.243541+0000","last_clean":"2024-04-18T22:22:33.243541+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:22:33.243541+0000","last_undegraded":"2024-04-18T22:22:33.243541+0000","last_fullsized":"2024-04-18T22:22:33.243541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242810+0000","last_change":"2024-04-18T22:22:32.146390+0000","last_active":"2024-04-18T22:22:33.242810+0000","last_peered":"2024-04-18T22:22:33.242810+0000","last_clean":"2024-04-18T22:22:33.242810+0000","last_became_active":"2024-04-18T22:22:32.145054+0000","last_became_peered":"2024-04-18T22:22:32.145054+0000","last_unstale":"2024-04-18T22:22:33.242810+0000","last_undegraded":"2024-04-18T22:22:33.242810+0000","last_fullsized":"2024-04-18T22:22:33.242810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:07:24.823488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127808+0000","last_change":"2024-04-18T22:22:32.132408+0000","last_active":"2024-04-18T22:22:33.127808+0000","last_peered":"2024-04-18T22:22:33.127808+0000","last_clean":"2024-04-18T22:22:33.127808+0000","last_became_active":"2024-04-18T22:22:32.131044+0000","last_became_peered":"2024-04-18T22:22:32.131044+0000","last_unstale":"2024-04-18T22:22:33.127808+0000","last_undegraded":"2024-04-18T22:22:33.127808+0000","last_fullsized":"2024-04-18T22:22:33.127808+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242233+0000","last_change":"2024-04-18T22:22:32.148879+0000","last_active":"2024-04-18T22:22:33.242233+0000","last_peered":"2024-04-18T22:22:33.242233+0000","last_clean":"2024-04-18T22:22:33.242233+0000","last_became_active":"2024-04-18T22:22:32.148070+0000","last_became_peered":"2024-04-18T22:22:32.148070+0000","last_unstale":"2024-04-18T22:22:33.242233+0000","last_undegraded":"2024-04-18T22:22:33.242233+0000","last_fullsized":"2024-04-18T22:22:33.242233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126582+0000","last_change":"2024-04-18T22:22:32.141405+0000","last_active":"2024-04-18T22:22:33.126582+0000","last_peered":"2024-04-18T22:22:33.126582+0000","last_clean":"2024-04-18T22:22:33.126582+0000","last_became_active":"2024-04-18T22:22:32.140369+0000","last_became_peered":"2024-04-18T22:22:32.140369+0000","last_unstale":"2024-04-18T22:22:33.126582+0000","last_undegraded":"2024-04-18T22:22:33.126582+0000","last_fullsized":"2024-04-18T22:22:33.126582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241205+0000","last_change":"2024-04-18T22:22:32.149208+0000","last_active":"2024-04-18T22:22:33.241205+0000","last_peered":"2024-04-18T22:22:33.241205+0000","last_clean":"2024-04-18T22:22:33.241205+0000","last_became_active":"2024-04-18T22:22:32.148344+0000","last_became_peered":"2024-04-18T22:22:32.148344+0000","last_unstale":"2024-04-18T22:22:33.241205+0000","last_undegraded":"2024-04-18T22:22:33.241205+0000","last_fullsized":"2024-04-18T22:22:33.241205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241714+0000","last_change":"2024-04-18T22:22:32.136695+0000","last_active":"2024-04-18T22:22:33.241714+0000","last_peered":"2024-04-18T22:22:33.241714+0000","last_clean":"2024-04-18T22:22:33.241714+0000","last_became_active":"2024-04-18T22:22:32.135901+0000","last_became_peered":"2024-04-18T22:22:32.135901+0000","last_unstale":"2024-04-18T22:22:33.241714+0000","last_undegraded":"2024-04-18T22:22:33.241714+0000","last_fullsized":"2024-04-18T22:22:33.241714+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127706+0000","last_change":"2024-04-18T22:22:32.144946+0000","last_active":"2024-04-18T22:22:33.127706+0000","last_peered":"2024-04-18T22:22:33.127706+0000","last_clean":"2024-04-18T22:22:33.127706+0000","last_became_active":"2024-04-18T22:22:32.143032+0000","last_became_peered":"2024-04-18T22:22:32.143032+0000","last_unstale":"2024-04-18T22:22:33.127706+0000","last_undegraded":"2024-04-18T22:22:33.127706+0000","last_fullsized":"2024-04-18T22:22:33.127706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242608+0000","last_change":"2024-04-18T22:22:31.145383+0000","last_active":"2024-04-18T22:22:33.242608+0000","last_peered":"2024-04-18T22:22:33.242608+0000","last_clean":"2024-04-18T22:22:33.242608+0000","last_became_active":"2024-04-18T22:22:31.143178+0000","last_became_peered":"2024-04-18T22:22:31.143178+0000","last_unstale":"2024-04-18T22:22:33.242608+0000","last_undegraded":"2024-04-18T22:22:33.242608+0000","last_fullsized":"2024-04-18T22:22:33.242608+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128208+0000","last_change":"2024-04-18T22:22:32.146608+0000","last_active":"2024-04-18T22:22:33.128208+0000","last_peered":"2024-04-18T22:22:33.128208+0000","last_clean":"2024-04-18T22:22:33.128208+0000","last_became_active":"2024-04-18T22:22:32.145761+0000","last_became_peered":"2024-04-18T22:22:32.145761+0000","last_unstale":"2024-04-18T22:22:33.128208+0000","last_undegraded":"2024-04-18T22:22:33.128208+0000","last_fullsized":"2024-04-18T22:22:33.128208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242107+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:22:33.242107+0000","last_peered":"2024-04-18T22:22:33.242107+0000","last_clean":"2024-04-18T22:22:33.242107+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:22:33.242107+0000","last_undegraded":"2024-04-18T22:22:33.242107+0000","last_fullsized":"2024-04-18T22:22:33.242107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243862+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:22:33.243862+0000","last_peered":"2024-04-18T22:22:33.243862+0000","last_clean":"2024-04-18T22:22:33.243862+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:22:33.243862+0000","last_undegraded":"2024-04-18T22:22:33.243862+0000","last_fullsized":"2024-04-18T22:22:33.243862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126186+0000","last_change":"2024-04-18T22:22:31.136552+0000","last_active":"2024-04-18T22:22:33.126186+0000","last_peered":"2024-04-18T22:22:33.126186+0000","last_clean":"2024-04-18T22:22:33.126186+0000","last_became_active":"2024-04-18T22:22:31.134824+0000","last_became_peered":"2024-04-18T22:22:31.134824+0000","last_unstale":"2024-04-18T22:22:33.126186+0000","last_undegraded":"2024-04-18T22:22:33.126186+0000","last_fullsized":"2024-04-18T22:22:33.126186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:07:17.380852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243033+0000","last_change":"2024-04-18T22:22:32.145785+0000","last_active":"2024-04-18T22:22:33.243033+0000","last_peered":"2024-04-18T22:22:33.243033+0000","last_clean":"2024-04-18T22:22:33.243033+0000","last_became_active":"2024-04-18T22:22:32.145166+0000","last_became_peered":"2024-04-18T22:22:32.145166+0000","last_unstale":"2024-04-18T22:22:33.243033+0000","last_undegraded":"2024-04-18T22:22:33.243033+0000","last_fullsized":"2024-04-18T22:22:33.243033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240502+0000","last_change":"2024-04-18T22:22:31.144229+0000","last_active":"2024-04-18T22:22:33.240502+0000","last_peered":"2024-04-18T22:22:33.240502+0000","last_clean":"2024-04-18T22:22:33.240502+0000","last_became_active":"2024-04-18T22:22:31.143597+0000","last_became_peered":"2024-04-18T22:22:31.143597+0000","last_unstale":"2024-04-18T22:22:33.240502+0000","last_undegraded":"2024-04-18T22:22:33.240502+0000","last_fullsized":"2024-04-18T22:22:33.240502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242009+0000","last_change":"2024-04-18T22:22:31.144343+0000","last_active":"2024-04-18T22:22:33.242009+0000","last_peered":"2024-04-18T22:22:33.242009+0000","last_clean":"2024-04-18T22:22:33.242009+0000","last_became_active":"2024-04-18T22:22:31.143325+0000","last_became_peered":"2024-04-18T22:22:31.143325+0000","last_unstale":"2024-04-18T22:22:33.242009+0000","last_undegraded":"2024-04-18T22:22:33.242009+0000","last_fullsized":"2024-04-18T22:22:33.242009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125867+0000","last_change":"2024-04-18T22:22:32.137726+0000","last_active":"2024-04-18T22:22:33.125867+0000","last_peered":"2024-04-18T22:22:33.125867+0000","last_clean":"2024-04-18T22:22:33.125867+0000","last_became_active":"2024-04-18T22:22:32.135216+0000","last_became_peered":"2024-04-18T22:22:32.135216+0000","last_unstale":"2024-04-18T22:22:33.125867+0000","last_undegraded":"2024-04-18T22:22:33.125867+0000","last_fullsized":"2024-04-18T22:22:33.125867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:01:43.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128180+0000","last_change":"2024-04-18T22:22:31.131784+0000","last_active":"2024-04-18T22:22:33.128180+0000","last_peered":"2024-04-18T22:22:33.128180+0000","last_clean":"2024-04-18T22:22:33.128180+0000","last_became_active":"2024-04-18T22:22:31.131002+0000","last_became_peered":"2024-04-18T22:22:31.131002+0000","last_unstale":"2024-04-18T22:22:33.128180+0000","last_undegraded":"2024-04-18T22:22:33.128180+0000","last_fullsized":"2024-04-18T22:22:33.128180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125897+0000","last_change":"2024-04-18T22:22:32.145000+0000","last_active":"2024-04-18T22:22:33.125897+0000","last_peered":"2024-04-18T22:22:33.125897+0000","last_clean":"2024-04-18T22:22:33.125897+0000","last_became_active":"2024-04-18T22:22:32.143522+0000","last_became_peered":"2024-04-18T22:22:32.143522+0000","last_unstale":"2024-04-18T22:22:33.125897+0000","last_undegraded":"2024-04-18T22:22:33.125897+0000","last_fullsized":"2024-04-18T22:22:33.125897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:26.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.144234+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.143532+0000","last_became_peered":"2024-04-18T22:22:31.143532+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:41.528679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241436+0000","last_change":"2024-04-18T22:22:32.135717+0000","last_active":"2024-04-18T22:22:33.241436+0000","last_peered":"2024-04-18T22:22:33.241436+0000","last_clean":"2024-04-18T22:22:33.241436+0000","last_became_active":"2024-04-18T22:22:32.134770+0000","last_became_peered":"2024-04-18T22:22:32.134770+0000","last_unstale":"2024-04-18T22:22:33.241436+0000","last_undegraded":"2024-04-18T22:22:33.241436+0000","last_fullsized":"2024-04-18T22:22:33.241436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242840+0000","last_change":"2024-04-18T22:22:31.147439+0000","last_active":"2024-04-18T22:22:33.242840+0000","last_peered":"2024-04-18T22:22:33.242840+0000","last_clean":"2024-04-18T22:22:33.242840+0000","last_became_active":"2024-04-18T22:22:31.146264+0000","last_became_peered":"2024-04-18T22:22:31.146264+0000","last_unstale":"2024-04-18T22:22:33.242840+0000","last_undegraded":"2024-04-18T22:22:33.242840+0000","last_fullsized":"2024-04-18T22:22:33.242840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242813+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:22:33.242813+0000","last_peered":"2024-04-18T22:22:33.242813+0000","last_clean":"2024-04-18T22:22:33.242813+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:22:33.242813+0000","last_undegraded":"2024-04-18T22:22:33.242813+0000","last_fullsized":"2024-04-18T22:22:33.242813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242737+0000","last_change":"2024-04-18T22:22:31.140451+0000","last_active":"2024-04-18T22:22:33.242737+0000","last_peered":"2024-04-18T22:22:33.242737+0000","last_clean":"2024-04-18T22:22:33.242737+0000","last_became_active":"2024-04-18T22:22:31.139655+0000","last_became_peered":"2024-04-18T22:22:31.139655+0000","last_unstale":"2024-04-18T22:22:33.242737+0000","last_undegraded":"2024-04-18T22:22:33.242737+0000","last_fullsized":"2024-04-18T22:22:33.242737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242161+0000","last_change":"2024-04-18T22:22:32.145869+0000","last_active":"2024-04-18T22:22:33.242161+0000","last_peered":"2024-04-18T22:22:33.242161+0000","last_clean":"2024-04-18T22:22:33.242161+0000","last_became_active":"2024-04-18T22:22:32.144117+0000","last_became_peered":"2024-04-18T22:22:32.144117+0000","last_unstale":"2024-04-18T22:22:33.242161+0000","last_undegraded":"2024-04-18T22:22:33.242161+0000","last_fullsized":"2024-04-18T22:22:33.242161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:39.186356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242361+0000","last_change":"2024-04-18T22:22:31.143197+0000","last_active":"2024-04-18T22:22:33.242361+0000","last_peered":"2024-04-18T22:22:33.242361+0000","last_clean":"2024-04-18T22:22:33.242361+0000","last_became_active":"2024-04-18T22:22:31.142393+0000","last_became_peered":"2024-04-18T22:22:31.142393+0000","last_unstale":"2024-04-18T22:22:33.242361+0000","last_undegraded":"2024-04-18T22:22:33.242361+0000","last_fullsized":"2024-04-18T22:22:33.242361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243118+0000","last_change":"2024-04-18T22:22:32.147396+0000","last_active":"2024-04-18T22:22:33.243118+0000","last_peered":"2024-04-18T22:22:33.243118+0000","last_clean":"2024-04-18T22:22:33.243118+0000","last_became_active":"2024-04-18T22:22:32.146327+0000","last_became_peered":"2024-04-18T22:22:32.146327+0000","last_unstale":"2024-04-18T22:22:33.243118+0000","last_undegraded":"2024-04-18T22:22:33.243118+0000","last_fullsized":"2024-04-18T22:22:33.243118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:16:31.884436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126713+0000","last_change":"2024-04-18T22:22:31.135058+0000","last_active":"2024-04-18T22:22:33.126713+0000","last_peered":"2024-04-18T22:22:33.126713+0000","last_clean":"2024-04-18T22:22:33.126713+0000","last_became_active":"2024-04-18T22:22:31.134213+0000","last_became_peered":"2024-04-18T22:22:31.134213+0000","last_unstale":"2024-04-18T22:22:33.126713+0000","last_undegraded":"2024-04-18T22:22:33.126713+0000","last_fullsized":"2024-04-18T22:22:33.126713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:35.837015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241581+0000","last_change":"2024-04-18T22:22:32.144956+0000","last_active":"2024-04-18T22:22:33.241581+0000","last_peered":"2024-04-18T22:22:33.241581+0000","last_clean":"2024-04-18T22:22:33.241581+0000","last_became_active":"2024-04-18T22:22:32.144171+0000","last_became_peered":"2024-04-18T22:22:32.144171+0000","last_unstale":"2024-04-18T22:22:33.241581+0000","last_undegraded":"2024-04-18T22:22:33.241581+0000","last_fullsized":"2024-04-18T22:22:33.241581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126257+0000","last_change":"2024-04-18T22:22:31.129222+0000","last_active":"2024-04-18T22:22:33.126257+0000","last_peered":"2024-04-18T22:22:33.126257+0000","last_clean":"2024-04-18T22:22:33.126257+0000","last_became_active":"2024-04-18T22:22:31.128618+0000","last_became_peered":"2024-04-18T22:22:31.128618+0000","last_unstale":"2024-04-18T22:22:33.126257+0000","last_undegraded":"2024-04-18T22:22:33.126257+0000","last_fullsized":"2024-04-18T22:22:33.126257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:58:49.687595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243356+0000","last_change":"2024-04-18T22:22:32.150911+0000","last_active":"2024-04-18T22:22:33.243356+0000","last_peered":"2024-04-18T22:22:33.243356+0000","last_clean":"2024-04-18T22:22:33.243356+0000","last_became_active":"2024-04-18T22:22:32.149188+0000","last_became_peered":"2024-04-18T22:22:32.149188+0000","last_unstale":"2024-04-18T22:22:33.243356+0000","last_undegraded":"2024-04-18T22:22:33.243356+0000","last_fullsized":"2024-04-18T22:22:33.243356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":222,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242605+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:33.242605+0000","last_peered":"2024-04-18T22:22:33.242605+0000","last_clean":"2024-04-18T22:22:33.242605+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:33.242605+0000","last_undegraded":"2024-04-18T22:22:33.242605+0000","last_fullsized":"2024-04-18T22:22:33.242605+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242816+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:33.242816+0000","last_peered":"2024-04-18T22:22:33.242816+0000","last_clean":"2024-04-18T22:22:33.242816+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:33.242816+0000","last_undegraded":"2024-04-18T22:22:33.242816+0000","last_fullsized":"2024-04-18T22:22:33.242816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242531+0000","last_change":"2024-04-18T22:22:31.128030+0000","last_active":"2024-04-18T22:22:33.242531+0000","last_peered":"2024-04-18T22:22:33.242531+0000","last_clean":"2024-04-18T22:22:33.242531+0000","last_became_active":"2024-04-18T22:22:31.126939+0000","last_became_peered":"2024-04-18T22:22:31.126939+0000","last_unstale":"2024-04-18T22:22:33.242531+0000","last_undegraded":"2024-04-18T22:22:33.242531+0000","last_fullsized":"2024-04-18T22:22:33.242531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243540+0000","last_change":"2024-04-18T22:22:32.153417+0000","last_active":"2024-04-18T22:22:33.243540+0000","last_peered":"2024-04-18T22:22:33.243540+0000","last_clean":"2024-04-18T22:22:33.243540+0000","last_became_active":"2024-04-18T22:22:32.152653+0000","last_became_peered":"2024-04-18T22:22:32.152653+0000","last_unstale":"2024-04-18T22:22:33.243540+0000","last_undegraded":"2024-04-18T22:22:33.243540+0000","last_fullsized":"2024-04-18T22:22:33.243540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:49:12.269993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242307+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:33.242307+0000","last_peered":"2024-04-18T22:22:33.242307+0000","last_clean":"2024-04-18T22:22:33.242307+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:33.242307+0000","last_undegraded":"2024-04-18T22:22:33.242307+0000","last_fullsized":"2024-04-18T22:22:33.242307+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242019+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:22:33.242019+0000","last_peered":"2024-04-18T22:22:33.242019+0000","last_clean":"2024-04-18T22:22:33.242019+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:22:33.242019+0000","last_undegraded":"2024-04-18T22:22:33.242019+0000","last_fullsized":"2024-04-18T22:22:33.242019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127898+0000","last_change":"2024-04-18T22:22:32.147651+0000","last_active":"2024-04-18T22:22:33.127898+0000","last_peered":"2024-04-18T22:22:33.127898+0000","last_clean":"2024-04-18T22:22:33.127898+0000","last_became_active":"2024-04-18T22:22:32.143774+0000","last_became_peered":"2024-04-18T22:22:32.143774+0000","last_unstale":"2024-04-18T22:22:33.127898+0000","last_undegraded":"2024-04-18T22:22:33.127898+0000","last_fullsized":"2024-04-18T22:22:33.127898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242167+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:33.242167+0000","last_peered":"2024-04-18T22:22:33.242167+0000","last_clean":"2024-04-18T22:22:33.242167+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:33.242167+0000","last_undegraded":"2024-04-18T22:22:33.242167+0000","last_fullsized":"2024-04-18T22:22:33.242167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126710+0000","last_change":"2024-04-18T22:22:31.132958+0000","last_active":"2024-04-18T22:22:33.126710+0000","last_peered":"2024-04-18T22:22:33.126710+0000","last_clean":"2024-04-18T22:22:33.126710+0000","last_became_active":"2024-04-18T22:22:31.132186+0000","last_became_peered":"2024-04-18T22:22:31.132186+0000","last_unstale":"2024-04-18T22:22:33.126710+0000","last_undegraded":"2024-04-18T22:22:33.126710+0000","last_fullsized":"2024-04-18T22:22:33.126710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243413+0000","last_change":"2024-04-18T22:22:32.150208+0000","last_active":"2024-04-18T22:22:33.243413+0000","last_peered":"2024-04-18T22:22:33.243413+0000","last_clean":"2024-04-18T22:22:33.243413+0000","last_became_active":"2024-04-18T22:22:32.148155+0000","last_became_peered":"2024-04-18T22:22:32.148155+0000","last_unstale":"2024-04-18T22:22:33.243413+0000","last_undegraded":"2024-04-18T22:22:33.243413+0000","last_fullsized":"2024-04-18T22:22:33.243413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241760+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:33.241760+0000","last_peered":"2024-04-18T22:22:33.241760+0000","last_clean":"2024-04-18T22:22:33.241760+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:33.241760+0000","last_undegraded":"2024-04-18T22:22:33.241760+0000","last_fullsized":"2024-04-18T22:22:33.241760+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242877+0000","last_change":"2024-04-18T22:22:31.146898+0000","last_active":"2024-04-18T22:22:33.242877+0000","last_peered":"2024-04-18T22:22:33.242877+0000","last_clean":"2024-04-18T22:22:33.242877+0000","last_became_active":"2024-04-18T22:22:31.146071+0000","last_became_peered":"2024-04-18T22:22:31.146071+0000","last_unstale":"2024-04-18T22:22:33.242877+0000","last_undegraded":"2024-04-18T22:22:33.242877+0000","last_fullsized":"2024-04-18T22:22:33.242877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243542+0000","last_change":"2024-04-18T22:22:32.147890+0000","last_active":"2024-04-18T22:22:33.243542+0000","last_peered":"2024-04-18T22:22:33.243542+0000","last_clean":"2024-04-18T22:22:33.243542+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.243542+0000","last_undegraded":"2024-04-18T22:22:33.243542+0000","last_fullsized":"2024-04-18T22:22:33.243542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243666+0000","last_change":"2024-04-18T22:22:31.146681+0000","last_active":"2024-04-18T22:22:33.243666+0000","last_peered":"2024-04-18T22:22:33.243666+0000","last_clean":"2024-04-18T22:22:33.243666+0000","last_became_active":"2024-04-18T22:22:31.145764+0000","last_became_peered":"2024-04-18T22:22:31.145764+0000","last_unstale":"2024-04-18T22:22:33.243666+0000","last_undegraded":"2024-04-18T22:22:33.243666+0000","last_fullsized":"2024-04-18T22:22:33.243666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:33.679846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243510+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:33.243510+0000","last_peered":"2024-04-18T22:22:33.243510+0000","last_clean":"2024-04-18T22:22:33.243510+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:33.243510+0000","last_undegraded":"2024-04-18T22:22:33.243510+0000","last_fullsized":"2024-04-18T22:22:33.243510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127522+0000","last_change":"2024-04-18T22:22:32.127971+0000","last_active":"2024-04-18T22:22:33.127522+0000","last_peered":"2024-04-18T22:22:33.127522+0000","last_clean":"2024-04-18T22:22:33.127522+0000","last_became_active":"2024-04-18T22:22:32.126518+0000","last_became_peered":"2024-04-18T22:22:32.126518+0000","last_unstale":"2024-04-18T22:22:33.127522+0000","last_undegraded":"2024-04-18T22:22:33.127522+0000","last_fullsized":"2024-04-18T22:22:33.127522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243415+0000","last_change":"2024-04-18T22:22:31.127787+0000","last_active":"2024-04-18T22:22:33.243415+0000","last_peered":"2024-04-18T22:22:33.243415+0000","last_clean":"2024-04-18T22:22:33.243415+0000","last_became_active":"2024-04-18T22:22:31.127173+0000","last_became_peered":"2024-04-18T22:22:31.127173+0000","last_unstale":"2024-04-18T22:22:33.243415+0000","last_undegraded":"2024-04-18T22:22:33.243415+0000","last_fullsized":"2024-04-18T22:22:33.243415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128505+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:33.128505+0000","last_peered":"2024-04-18T22:22:33.128505+0000","last_clean":"2024-04-18T22:22:33.128505+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:33.128505+0000","last_undegraded":"2024-04-18T22:22:33.128505+0000","last_fullsized":"2024-04-18T22:22:33.128505+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126910+0000","last_change":"2024-04-18T22:22:32.136161+0000","last_active":"2024-04-18T22:22:33.126910+0000","last_peered":"2024-04-18T22:22:33.126910+0000","last_clean":"2024-04-18T22:22:33.126910+0000","last_became_active":"2024-04-18T22:22:32.135253+0000","last_became_peered":"2024-04-18T22:22:32.135253+0000","last_unstale":"2024-04-18T22:22:33.126910+0000","last_undegraded":"2024-04-18T22:22:33.126910+0000","last_fullsized":"2024-04-18T22:22:33.126910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244171+0000","last_change":"2024-04-18T22:22:31.136319+0000","last_active":"2024-04-18T22:22:33.244171+0000","last_peered":"2024-04-18T22:22:33.244171+0000","last_clean":"2024-04-18T22:22:33.244171+0000","last_became_active":"2024-04-18T22:22:31.135689+0000","last_became_peered":"2024-04-18T22:22:31.135689+0000","last_unstale":"2024-04-18T22:22:33.244171+0000","last_undegraded":"2024-04-18T22:22:33.244171+0000","last_fullsized":"2024-04-18T22:22:33.244171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244003+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:33.244003+0000","last_peered":"2024-04-18T22:22:33.244003+0000","last_clean":"2024-04-18T22:22:33.244003+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:33.244003+0000","last_undegraded":"2024-04-18T22:22:33.244003+0000","last_fullsized":"2024-04-18T22:22:33.244003+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242936+0000","last_change":"2024-04-18T22:22:32.144432+0000","last_active":"2024-04-18T22:22:33.242936+0000","last_peered":"2024-04-18T22:22:33.242936+0000","last_clean":"2024-04-18T22:22:33.242936+0000","last_became_active":"2024-04-18T22:22:32.142947+0000","last_became_peered":"2024-04-18T22:22:32.142947+0000","last_unstale":"2024-04-18T22:22:33.242936+0000","last_undegraded":"2024-04-18T22:22:33.242936+0000","last_fullsized":"2024-04-18T22:22:33.242936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:34.119061+0000","last_change":"2024-04-18T22:22:31.143202+0000","last_active":"2024-04-18T22:22:34.119061+0000","last_peered":"2024-04-18T22:22:34.119061+0000","last_clean":"2024-04-18T22:22:34.119061+0000","last_became_active":"2024-04-18T22:22:31.142465+0000","last_became_peered":"2024-04-18T22:22:31.142465+0000","last_unstale":"2024-04-18T22:22:34.119061+0000","last_undegraded":"2024-04-18T22:22:34.119061+0000","last_fullsized":"2024-04-18T22:22:34.119061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241961+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:33.241961+0000","last_peered":"2024-04-18T22:22:33.241961+0000","last_clean":"2024-04-18T22:22:33.241961+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:33.241961+0000","last_undegraded":"2024-04-18T22:22:33.241961+0000","last_fullsized":"2024-04-18T22:22:33.241961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125912+0000","last_change":"2024-04-18T22:22:32.132968+0000","last_active":"2024-04-18T22:22:33.125912+0000","last_peered":"2024-04-18T22:22:33.125912+0000","last_clean":"2024-04-18T22:22:33.125912+0000","last_became_active":"2024-04-18T22:22:32.131596+0000","last_became_peered":"2024-04-18T22:22:32.131596+0000","last_unstale":"2024-04-18T22:22:33.125912+0000","last_undegraded":"2024-04-18T22:22:33.125912+0000","last_fullsized":"2024-04-18T22:22:33.125912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241097+0000","last_change":"2024-04-18T22:22:32.145633+0000","last_active":"2024-04-18T22:22:33.241097+0000","last_peered":"2024-04-18T22:22:33.241097+0000","last_clean":"2024-04-18T22:22:33.241097+0000","last_became_active":"2024-04-18T22:22:32.142868+0000","last_became_peered":"2024-04-18T22:22:32.142868+0000","last_unstale":"2024-04-18T22:22:33.241097+0000","last_undegraded":"2024-04-18T22:22:33.241097+0000","last_fullsized":"2024-04-18T22:22:33.241097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242663+0000","last_change":"2024-04-18T22:22:31.144413+0000","last_active":"2024-04-18T22:22:33.242663+0000","last_peered":"2024-04-18T22:22:33.242663+0000","last_clean":"2024-04-18T22:22:33.242663+0000","last_became_active":"2024-04-18T22:22:31.143704+0000","last_became_peered":"2024-04-18T22:22:31.143704+0000","last_unstale":"2024-04-18T22:22:33.242663+0000","last_undegraded":"2024-04-18T22:22:33.242663+0000","last_fullsized":"2024-04-18T22:22:33.242663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241992+0000","last_change":"2024-04-18T22:22:32.148407+0000","last_active":"2024-04-18T22:22:33.241992+0000","last_peered":"2024-04-18T22:22:33.241992+0000","last_clean":"2024-04-18T22:22:33.241992+0000","last_became_active":"2024-04-18T22:22:32.147033+0000","last_became_peered":"2024-04-18T22:22:32.147033+0000","last_unstale":"2024-04-18T22:22:33.241992+0000","last_undegraded":"2024-04-18T22:22:33.241992+0000","last_fullsized":"2024-04-18T22:22:33.241992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128409+0000","last_change":"2024-04-18T22:22:31.133306+0000","last_active":"2024-04-18T22:22:33.128409+0000","last_peered":"2024-04-18T22:22:33.128409+0000","last_clean":"2024-04-18T22:22:33.128409+0000","last_became_active":"2024-04-18T22:22:31.132648+0000","last_became_peered":"2024-04-18T22:22:31.132648+0000","last_unstale":"2024-04-18T22:22:33.128409+0000","last_undegraded":"2024-04-18T22:22:33.128409+0000","last_fullsized":"2024-04-18T22:22:33.128409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242551+0000","last_change":"2024-04-18T22:22:31.146827+0000","last_active":"2024-04-18T22:22:33.242551+0000","last_peered":"2024-04-18T22:22:33.242551+0000","last_clean":"2024-04-18T22:22:33.242551+0000","last_became_active":"2024-04-18T22:22:31.146063+0000","last_became_peered":"2024-04-18T22:22:31.146063+0000","last_unstale":"2024-04-18T22:22:33.242551+0000","last_undegraded":"2024-04-18T22:22:33.242551+0000","last_fullsized":"2024-04-18T22:22:33.242551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241955+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:22:33.241955+0000","last_peered":"2024-04-18T22:22:33.241955+0000","last_clean":"2024-04-18T22:22:33.241955+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:22:33.241955+0000","last_undegraded":"2024-04-18T22:22:33.241955+0000","last_fullsized":"2024-04-18T22:22:33.241955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243856+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:22:33.243856+0000","last_peered":"2024-04-18T22:22:33.243856+0000","last_clean":"2024-04-18T22:22:33.243856+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:22:33.243856+0000","last_undegraded":"2024-04-18T22:22:33.243856+0000","last_fullsized":"2024-04-18T22:22:33.243856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241179+0000","last_change":"2024-04-18T22:22:32.146927+0000","last_active":"2024-04-18T22:22:33.241179+0000","last_peered":"2024-04-18T22:22:33.241179+0000","last_clean":"2024-04-18T22:22:33.241179+0000","last_became_active":"2024-04-18T22:22:32.146040+0000","last_became_peered":"2024-04-18T22:22:32.146040+0000","last_unstale":"2024-04-18T22:22:33.241179+0000","last_undegraded":"2024-04-18T22:22:33.241179+0000","last_fullsized":"2024-04-18T22:22:33.241179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128951+0000","last_change":"2024-04-18T22:22:32.139063+0000","last_active":"2024-04-18T22:22:33.128951+0000","last_peered":"2024-04-18T22:22:33.128951+0000","last_clean":"2024-04-18T22:22:33.128951+0000","last_became_active":"2024-04-18T22:22:32.138016+0000","last_became_peered":"2024-04-18T22:22:32.138016+0000","last_unstale":"2024-04-18T22:22:33.128951+0000","last_undegraded":"2024-04-18T22:22:33.128951+0000","last_fullsized":"2024-04-18T22:22:33.128951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241615+0000","last_change":"2024-04-18T22:22:31.130452+0000","last_active":"2024-04-18T22:22:33.241615+0000","last_peered":"2024-04-18T22:22:33.241615+0000","last_clean":"2024-04-18T22:22:33.241615+0000","last_became_active":"2024-04-18T22:22:31.129555+0000","last_became_peered":"2024-04-18T22:22:31.129555+0000","last_unstale":"2024-04-18T22:22:33.241615+0000","last_undegraded":"2024-04-18T22:22:33.241615+0000","last_fullsized":"2024-04-18T22:22:33.241615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243327+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:22:33.243327+0000","last_peered":"2024-04-18T22:22:33.243327+0000","last_clean":"2024-04-18T22:22:33.243327+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:22:33.243327+0000","last_undegraded":"2024-04-18T22:22:33.243327+0000","last_fullsized":"2024-04-18T22:22:33.243327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241696+0000","last_change":"2024-04-18T22:22:31.133021+0000","last_active":"2024-04-18T22:22:33.241696+0000","last_peered":"2024-04-18T22:22:33.241696+0000","last_clean":"2024-04-18T22:22:33.241696+0000","last_became_active":"2024-04-18T22:22:31.132544+0000","last_became_peered":"2024-04-18T22:22:31.132544+0000","last_unstale":"2024-04-18T22:22:33.241696+0000","last_undegraded":"2024-04-18T22:22:33.241696+0000","last_fullsized":"2024-04-18T22:22:33.241696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127774+0000","last_change":"2024-04-18T22:22:32.138696+0000","last_active":"2024-04-18T22:22:33.127774+0000","last_peered":"2024-04-18T22:22:33.127774+0000","last_clean":"2024-04-18T22:22:33.127774+0000","last_became_active":"2024-04-18T22:22:32.137900+0000","last_became_peered":"2024-04-18T22:22:32.137900+0000","last_unstale":"2024-04-18T22:22:33.127774+0000","last_undegraded":"2024-04-18T22:22:33.127774+0000","last_fullsized":"2024-04-18T22:22:33.127774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126866+0000","last_change":"2024-04-18T22:22:31.133358+0000","last_active":"2024-04-18T22:22:33.126866+0000","last_peered":"2024-04-18T22:22:33.126866+0000","last_clean":"2024-04-18T22:22:33.126866+0000","last_became_active":"2024-04-18T22:22:31.132642+0000","last_became_peered":"2024-04-18T22:22:31.132642+0000","last_unstale":"2024-04-18T22:22:33.126866+0000","last_undegraded":"2024-04-18T22:22:33.126866+0000","last_fullsized":"2024-04-18T22:22:33.126866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127236+0000","last_change":"2024-04-18T22:22:32.146126+0000","last_active":"2024-04-18T22:22:33.127236+0000","last_peered":"2024-04-18T22:22:33.127236+0000","last_clean":"2024-04-18T22:22:33.127236+0000","last_became_active":"2024-04-18T22:22:32.144083+0000","last_became_peered":"2024-04-18T22:22:32.144083+0000","last_unstale":"2024-04-18T22:22:33.127236+0000","last_undegraded":"2024-04-18T22:22:33.127236+0000","last_fullsized":"2024-04-18T22:22:33.127236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127648+0000","last_change":"2024-04-18T22:22:31.136636+0000","last_active":"2024-04-18T22:22:33.127648+0000","last_peered":"2024-04-18T22:22:33.127648+0000","last_clean":"2024-04-18T22:22:33.127648+0000","last_became_active":"2024-04-18T22:22:31.135039+0000","last_became_peered":"2024-04-18T22:22:31.135039+0000","last_unstale":"2024-04-18T22:22:33.127648+0000","last_undegraded":"2024-04-18T22:22:33.127648+0000","last_fullsized":"2024-04-18T22:22:33.127648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.293851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240793+0000","last_change":"2024-04-18T22:22:32.134794+0000","last_active":"2024-04-18T22:22:33.240793+0000","last_peered":"2024-04-18T22:22:33.240793+0000","last_clean":"2024-04-18T22:22:33.240793+0000","last_became_active":"2024-04-18T22:22:32.133858+0000","last_became_peered":"2024-04-18T22:22:32.133858+0000","last_unstale":"2024-04-18T22:22:33.240793+0000","last_undegraded":"2024-04-18T22:22:33.240793+0000","last_fullsized":"2024-04-18T22:22:33.240793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242444+0000","last_change":"2024-04-18T22:22:31.147456+0000","last_active":"2024-04-18T22:22:33.242444+0000","last_peered":"2024-04-18T22:22:33.242444+0000","last_clean":"2024-04-18T22:22:33.242444+0000","last_became_active":"2024-04-18T22:22:31.145624+0000","last_became_peered":"2024-04-18T22:22:31.145624+0000","last_unstale":"2024-04-18T22:22:33.242444+0000","last_undegraded":"2024-04-18T22:22:33.242444+0000","last_fullsized":"2024-04-18T22:22:33.242444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241579+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.241579+0000","last_peered":"2024-04-18T22:22:33.241579+0000","last_clean":"2024-04-18T22:22:33.241579+0000","last_became_active":"2024-04-18T22:22:32.143184+0000","last_became_peered":"2024-04-18T22:22:32.143184+0000","last_unstale":"2024-04-18T22:22:33.241579+0000","last_undegraded":"2024-04-18T22:22:33.241579+0000","last_fullsized":"2024-04-18T22:22:33.241579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128174+0000","last_change":"2024-04-18T22:22:31.138492+0000","last_active":"2024-04-18T22:22:33.128174+0000","last_peered":"2024-04-18T22:22:33.128174+0000","last_clean":"2024-04-18T22:22:33.128174+0000","last_became_active":"2024-04-18T22:22:31.134646+0000","last_became_peered":"2024-04-18T22:22:31.134646+0000","last_unstale":"2024-04-18T22:22:33.128174+0000","last_undegraded":"2024-04-18T22:22:33.128174+0000","last_fullsized":"2024-04-18T22:22:33.128174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127146+0000","last_change":"2024-04-18T22:22:32.137473+0000","last_active":"2024-04-18T22:22:33.127146+0000","last_peered":"2024-04-18T22:22:33.127146+0000","last_clean":"2024-04-18T22:22:33.127146+0000","last_became_active":"2024-04-18T22:22:32.136025+0000","last_became_peered":"2024-04-18T22:22:32.136025+0000","last_unstale":"2024-04-18T22:22:33.127146+0000","last_undegraded":"2024-04-18T22:22:33.127146+0000","last_fullsized":"2024-04-18T22:22:33.127146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243824+0000","last_change":"2024-04-18T22:22:31.130827+0000","last_active":"2024-04-18T22:22:33.243824+0000","last_peered":"2024-04-18T22:22:33.243824+0000","last_clean":"2024-04-18T22:22:33.243824+0000","last_became_active":"2024-04-18T22:22:31.129728+0000","last_became_peered":"2024-04-18T22:22:31.129728+0000","last_unstale":"2024-04-18T22:22:33.243824+0000","last_undegraded":"2024-04-18T22:22:33.243824+0000","last_fullsized":"2024-04-18T22:22:33.243824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242152+0000","last_change":"2024-04-18T22:22:32.144325+0000","last_active":"2024-04-18T22:22:33.242152+0000","last_peered":"2024-04-18T22:22:33.242152+0000","last_clean":"2024-04-18T22:22:33.242152+0000","last_became_active":"2024-04-18T22:22:32.143495+0000","last_became_peered":"2024-04-18T22:22:32.143495+0000","last_unstale":"2024-04-18T22:22:33.242152+0000","last_undegraded":"2024-04-18T22:22:33.242152+0000","last_fullsized":"2024-04-18T22:22:33.242152+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.142640+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.141819+0000","last_became_peered":"2024-04-18T22:22:31.141819+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738378,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":245760,"data_stored":647890,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":647274,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":55728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:52.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:22:52.587 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:22:52.589 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:22:52.605 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-18T22:22:50.833692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":90020,"kb_used_data":632,"kb_used_omap":4,"kb_used_meta":89339,"kb_avail":283025500,"statfs":{"total":289910292480,"available":289818112000,"internally_reserved":0,"allocated":647168,"data_stored":1350892,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":91483471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003563"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128432+0000","last_change":"2024-04-18T22:22:32.145997+0000","last_active":"2024-04-18T22:22:33.128432+0000","last_peered":"2024-04-18T22:22:33.128432+0000","last_clean":"2024-04-18T22:22:33.128432+0000","last_became_active":"2024-04-18T22:22:32.143852+0000","last_became_peered":"2024-04-18T22:22:32.143852+0000","last_unstale":"2024-04-18T22:22:33.128432+0000","last_undegraded":"2024-04-18T22:22:33.128432+0000","last_fullsized":"2024-04-18T22:22:33.128432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126921+0000","last_change":"2024-04-18T22:22:32.148788+0000","last_active":"2024-04-18T22:22:33.126921+0000","last_peered":"2024-04-18T22:22:33.126921+0000","last_clean":"2024-04-18T22:22:33.126921+0000","last_became_active":"2024-04-18T22:22:32.146781+0000","last_became_peered":"2024-04-18T22:22:32.146781+0000","last_unstale":"2024-04-18T22:22:33.126921+0000","last_undegraded":"2024-04-18T22:22:33.126921+0000","last_fullsized":"2024-04-18T22:22:33.126921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241643+0000","last_change":"2024-04-18T22:22:32.152874+0000","last_active":"2024-04-18T22:22:33.241643+0000","last_peered":"2024-04-18T22:22:33.241643+0000","last_clean":"2024-04-18T22:22:33.241643+0000","last_became_active":"2024-04-18T22:22:32.152154+0000","last_became_peered":"2024-04-18T22:22:32.152154+0000","last_unstale":"2024-04-18T22:22:33.241643+0000","last_undegraded":"2024-04-18T22:22:33.241643+0000","last_fullsized":"2024-04-18T22:22:33.241643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242573+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.242573+0000","last_peered":"2024-04-18T22:22:33.242573+0000","last_clean":"2024-04-18T22:22:33.242573+0000","last_became_active":"2024-04-18T22:22:32.142591+0000","last_became_peered":"2024-04-18T22:22:32.142591+0000","last_unstale":"2024-04-18T22:22:33.242573+0000","last_undegraded":"2024-04-18T22:22:33.242573+0000","last_fullsized":"2024-04-18T22:22:33.242573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:24:32.079309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127358+0000","last_change":"2024-04-18T22:22:32.136433+0000","last_active":"2024-04-18T22:22:33.127358+0000","last_peered":"2024-04-18T22:22:33.127358+0000","last_clean":"2024-04-18T22:22:33.127358+0000","last_became_active":"2024-04-18T22:22:32.135586+0000","last_became_peered":"2024-04-18T22:22:32.135586+0000","last_unstale":"2024-04-18T22:22:33.127358+0000","last_undegraded":"2024-04-18T22:22:33.127358+0000","last_fullsized":"2024-04-18T22:22:33.127358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240606+0000","last_change":"2024-04-18T22:22:32.134957+0000","last_active":"2024-04-18T22:22:33.240606+0000","last_peered":"2024-04-18T22:22:33.240606+0000","last_clean":"2024-04-18T22:22:33.240606+0000","last_became_active":"2024-04-18T22:22:32.133797+0000","last_became_peered":"2024-04-18T22:22:32.133797+0000","last_unstale":"2024-04-18T22:22:33.240606+0000","last_undegraded":"2024-04-18T22:22:33.240606+0000","last_fullsized":"2024-04-18T22:22:33.240606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240547+0000","last_change":"2024-04-18T22:22:32.133775+0000","last_active":"2024-04-18T22:22:33.240547+0000","last_peered":"2024-04-18T22:22:33.240547+0000","last_clean":"2024-04-18T22:22:33.240547+0000","last_became_active":"2024-04-18T22:22:32.132972+0000","last_became_peered":"2024-04-18T22:22:32.132972+0000","last_unstale":"2024-04-18T22:22:33.240547+0000","last_undegraded":"2024-04-18T22:22:33.240547+0000","last_fullsized":"2024-04-18T22:22:33.240547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.239986+0000","last_change":"2024-04-18T22:22:32.136411+0000","last_active":"2024-04-18T22:22:33.239986+0000","last_peered":"2024-04-18T22:22:33.239986+0000","last_clean":"2024-04-18T22:22:33.239986+0000","last_became_active":"2024-04-18T22:22:32.132809+0000","last_became_peered":"2024-04-18T22:22:32.132809+0000","last_unstale":"2024-04-18T22:22:33.239986+0000","last_undegraded":"2024-04-18T22:22:33.239986+0000","last_fullsized":"2024-04-18T22:22:33.239986+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242905+0000","last_change":"2024-04-18T22:22:32.150575+0000","last_active":"2024-04-18T22:22:33.242905+0000","last_peered":"2024-04-18T22:22:33.242905+0000","last_clean":"2024-04-18T22:22:33.242905+0000","last_became_active":"2024-04-18T22:22:32.149253+0000","last_became_peered":"2024-04-18T22:22:32.149253+0000","last_unstale":"2024-04-18T22:22:33.242905+0000","last_undegraded":"2024-04-18T22:22:33.242905+0000","last_fullsized":"2024-04-18T22:22:33.242905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127402+0000","last_change":"2024-04-18T22:22:32.148298+0000","last_active":"2024-04-18T22:22:33.127402+0000","last_peered":"2024-04-18T22:22:33.127402+0000","last_clean":"2024-04-18T22:22:33.127402+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.127402+0000","last_undegraded":"2024-04-18T22:22:33.127402+0000","last_fullsized":"2024-04-18T22:22:33.127402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242310+0000","last_change":"2024-04-18T22:22:32.133832+0000","last_active":"2024-04-18T22:22:33.242310+0000","last_peered":"2024-04-18T22:22:33.242310+0000","last_clean":"2024-04-18T22:22:33.242310+0000","last_became_active":"2024-04-18T22:22:32.132844+0000","last_became_peered":"2024-04-18T22:22:32.132844+0000","last_unstale":"2024-04-18T22:22:33.242310+0000","last_undegraded":"2024-04-18T22:22:33.242310+0000","last_fullsized":"2024-04-18T22:22:33.242310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:04:32.092674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126733+0000","last_change":"2024-04-18T22:22:32.140365+0000","last_active":"2024-04-18T22:22:33.126733+0000","last_peered":"2024-04-18T22:22:33.126733+0000","last_clean":"2024-04-18T22:22:33.126733+0000","last_became_active":"2024-04-18T22:22:32.138147+0000","last_became_peered":"2024-04-18T22:22:32.138147+0000","last_unstale":"2024-04-18T22:22:33.126733+0000","last_undegraded":"2024-04-18T22:22:33.126733+0000","last_fullsized":"2024-04-18T22:22:33.126733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127632+0000","last_change":"2024-04-18T22:22:32.146043+0000","last_active":"2024-04-18T22:22:33.127632+0000","last_peered":"2024-04-18T22:22:33.127632+0000","last_clean":"2024-04-18T22:22:33.127632+0000","last_became_active":"2024-04-18T22:22:32.144174+0000","last_became_peered":"2024-04-18T22:22:32.144174+0000","last_unstale":"2024-04-18T22:22:33.127632+0000","last_undegraded":"2024-04-18T22:22:33.127632+0000","last_fullsized":"2024-04-18T22:22:33.127632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240784+0000","last_change":"2024-04-18T22:22:32.142740+0000","last_active":"2024-04-18T22:22:33.240784+0000","last_peered":"2024-04-18T22:22:33.240784+0000","last_clean":"2024-04-18T22:22:33.240784+0000","last_became_active":"2024-04-18T22:22:32.142048+0000","last_became_peered":"2024-04-18T22:22:32.142048+0000","last_unstale":"2024-04-18T22:22:33.240784+0000","last_undegraded":"2024-04-18T22:22:33.240784+0000","last_fullsized":"2024-04-18T22:22:33.240784+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240758+0000","last_change":"2024-04-18T22:22:32.129635+0000","last_active":"2024-04-18T22:22:33.240758+0000","last_peered":"2024-04-18T22:22:33.240758+0000","last_clean":"2024-04-18T22:22:33.240758+0000","last_became_active":"2024-04-18T22:22:32.128909+0000","last_became_peered":"2024-04-18T22:22:32.128909+0000","last_unstale":"2024-04-18T22:22:33.240758+0000","last_undegraded":"2024-04-18T22:22:33.240758+0000","last_fullsized":"2024-04-18T22:22:33.240758+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240016+0000","last_change":"2024-04-18T22:22:32.135755+0000","last_active":"2024-04-18T22:22:33.240016+0000","last_peered":"2024-04-18T22:22:33.240016+0000","last_clean":"2024-04-18T22:22:33.240016+0000","last_became_active":"2024-04-18T22:22:32.134588+0000","last_became_peered":"2024-04-18T22:22:32.134588+0000","last_unstale":"2024-04-18T22:22:33.240016+0000","last_undegraded":"2024-04-18T22:22:33.240016+0000","last_fullsized":"2024-04-18T22:22:33.240016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244495+0000","last_change":"2024-04-18T22:22:32.150194+0000","last_active":"2024-04-18T22:22:33.244495+0000","last_peered":"2024-04-18T22:22:33.244495+0000","last_clean":"2024-04-18T22:22:33.244495+0000","last_became_active":"2024-04-18T22:22:32.147919+0000","last_became_peered":"2024-04-18T22:22:32.147919+0000","last_unstale":"2024-04-18T22:22:33.244495+0000","last_undegraded":"2024-04-18T22:22:33.244495+0000","last_fullsized":"2024-04-18T22:22:33.244495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127840+0000","last_change":"2024-04-18T22:22:32.147917+0000","last_active":"2024-04-18T22:22:33.127840+0000","last_peered":"2024-04-18T22:22:33.127840+0000","last_clean":"2024-04-18T22:22:33.127840+0000","last_became_active":"2024-04-18T22:22:32.145953+0000","last_became_peered":"2024-04-18T22:22:32.145953+0000","last_unstale":"2024-04-18T22:22:33.127840+0000","last_undegraded":"2024-04-18T22:22:33.127840+0000","last_fullsized":"2024-04-18T22:22:33.127840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242825+0000","last_change":"2024-04-18T22:22:32.154655+0000","last_active":"2024-04-18T22:22:33.242825+0000","last_peered":"2024-04-18T22:22:33.242825+0000","last_clean":"2024-04-18T22:22:33.242825+0000","last_became_active":"2024-04-18T22:22:32.153966+0000","last_became_peered":"2024-04-18T22:22:32.153966+0000","last_unstale":"2024-04-18T22:22:33.242825+0000","last_undegraded":"2024-04-18T22:22:33.242825+0000","last_fullsized":"2024-04-18T22:22:33.242825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241750+0000","last_change":"2024-04-18T22:22:32.144928+0000","last_active":"2024-04-18T22:22:33.241750+0000","last_peered":"2024-04-18T22:22:33.241750+0000","last_clean":"2024-04-18T22:22:33.241750+0000","last_became_active":"2024-04-18T22:22:32.143282+0000","last_became_peered":"2024-04-18T22:22:32.143282+0000","last_unstale":"2024-04-18T22:22:33.241750+0000","last_undegraded":"2024-04-18T22:22:33.241750+0000","last_fullsized":"2024-04-18T22:22:33.241750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:16:54.977900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242442+0000","last_change":"2024-04-18T22:22:32.135811+0000","last_active":"2024-04-18T22:22:33.242442+0000","last_peered":"2024-04-18T22:22:33.242442+0000","last_clean":"2024-04-18T22:22:33.242442+0000","last_became_active":"2024-04-18T22:22:32.133269+0000","last_became_peered":"2024-04-18T22:22:32.133269+0000","last_unstale":"2024-04-18T22:22:33.242442+0000","last_undegraded":"2024-04-18T22:22:33.242442+0000","last_fullsized":"2024-04-18T22:22:33.242442+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:54:20.135916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126145+0000","last_change":"2024-04-18T22:22:32.141500+0000","last_active":"2024-04-18T22:22:33.126145+0000","last_peered":"2024-04-18T22:22:33.126145+0000","last_clean":"2024-04-18T22:22:33.126145+0000","last_became_active":"2024-04-18T22:22:32.140691+0000","last_became_peered":"2024-04-18T22:22:32.140691+0000","last_unstale":"2024-04-18T22:22:33.126145+0000","last_undegraded":"2024-04-18T22:22:33.126145+0000","last_fullsized":"2024-04-18T22:22:33.126145+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240965+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:22:33.240965+0000","last_peered":"2024-04-18T22:22:33.240965+0000","last_clean":"2024-04-18T22:22:33.240965+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:22:33.240965+0000","last_undegraded":"2024-04-18T22:22:33.240965+0000","last_fullsized":"2024-04-18T22:22:33.240965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126203+0000","last_change":"2024-04-18T22:22:32.141378+0000","last_active":"2024-04-18T22:22:33.126203+0000","last_peered":"2024-04-18T22:22:33.126203+0000","last_clean":"2024-04-18T22:22:33.126203+0000","last_became_active":"2024-04-18T22:22:32.140448+0000","last_became_peered":"2024-04-18T22:22:32.140448+0000","last_unstale":"2024-04-18T22:22:33.126203+0000","last_undegraded":"2024-04-18T22:22:33.126203+0000","last_fullsized":"2024-04-18T22:22:33.126203+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128370+0000","last_change":"2024-04-18T22:22:32.138865+0000","last_active":"2024-04-18T22:22:33.128370+0000","last_peered":"2024-04-18T22:22:33.128370+0000","last_clean":"2024-04-18T22:22:33.128370+0000","last_became_active":"2024-04-18T22:22:32.137793+0000","last_became_peered":"2024-04-18T22:22:32.137793+0000","last_unstale":"2024-04-18T22:22:33.128370+0000","last_undegraded":"2024-04-18T22:22:33.128370+0000","last_fullsized":"2024-04-18T22:22:33.128370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243541+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:22:33.243541+0000","last_peered":"2024-04-18T22:22:33.243541+0000","last_clean":"2024-04-18T22:22:33.243541+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:22:33.243541+0000","last_undegraded":"2024-04-18T22:22:33.243541+0000","last_fullsized":"2024-04-18T22:22:33.243541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242810+0000","last_change":"2024-04-18T22:22:32.146390+0000","last_active":"2024-04-18T22:22:33.242810+0000","last_peered":"2024-04-18T22:22:33.242810+0000","last_clean":"2024-04-18T22:22:33.242810+0000","last_became_active":"2024-04-18T22:22:32.145054+0000","last_became_peered":"2024-04-18T22:22:32.145054+0000","last_unstale":"2024-04-18T22:22:33.242810+0000","last_undegraded":"2024-04-18T22:22:33.242810+0000","last_fullsized":"2024-04-18T22:22:33.242810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:07:24.823488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127808+0000","last_change":"2024-04-18T22:22:32.132408+0000","last_active":"2024-04-18T22:22:33.127808+0000","last_peered":"2024-04-18T22:22:33.127808+0000","last_clean":"2024-04-18T22:22:33.127808+0000","last_became_active":"2024-04-18T22:22:32.131044+0000","last_became_peered":"2024-04-18T22:22:32.131044+0000","last_unstale":"2024-04-18T22:22:33.127808+0000","last_undegraded":"2024-04-18T22:22:33.127808+0000","last_fullsized":"2024-04-18T22:22:33.127808+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242233+0000","last_change":"2024-04-18T22:22:32.148879+0000","last_active":"2024-04-18T22:22:33.242233+0000","last_peered":"2024-04-18T22:22:33.242233+0000","last_clean":"2024-04-18T22:22:33.242233+0000","last_became_active":"2024-04-18T22:22:32.148070+0000","last_became_peered":"2024-04-18T22:22:32.148070+0000","last_unstale":"2024-04-18T22:22:33.242233+0000","last_undegraded":"2024-04-18T22:22:33.242233+0000","last_fullsized":"2024-04-18T22:22:33.242233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126582+0000","last_change":"2024-04-18T22:22:32.141405+0000","last_active":"2024-04-18T22:22:33.126582+0000","last_peered":"2024-04-18T22:22:33.126582+0000","last_clean":"2024-04-18T22:22:33.126582+0000","last_became_active":"2024-04-18T22:22:32.140369+0000","last_became_peered":"2024-04-18T22:22:32.140369+0000","last_unstale":"2024-04-18T22:22:33.126582+0000","last_undegraded":"2024-04-18T22:22:33.126582+0000","last_fullsized":"2024-04-18T22:22:33.126582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241205+0000","last_change":"2024-04-18T22:22:32.149208+0000","last_active":"2024-04-18T22:22:33.241205+0000","last_peered":"2024-04-18T22:22:33.241205+0000","last_clean":"2024-04-18T22:22:33.241205+0000","last_became_active":"2024-04-18T22:22:32.148344+0000","last_became_peered":"2024-04-18T22:22:32.148344+0000","last_unstale":"2024-04-18T22:22:33.241205+0000","last_undegraded":"2024-04-18T22:22:33.241205+0000","last_fullsized":"2024-04-18T22:22:33.241205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:16.172404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241714+0000","last_change":"2024-04-18T22:22:32.136695+0000","last_active":"2024-04-18T22:22:33.241714+0000","last_peered":"2024-04-18T22:22:33.241714+0000","last_clean":"2024-04-18T22:22:33.241714+0000","last_became_active":"2024-04-18T22:22:32.135901+0000","last_became_peered":"2024-04-18T22:22:32.135901+0000","last_unstale":"2024-04-18T22:22:33.241714+0000","last_undegraded":"2024-04-18T22:22:33.241714+0000","last_fullsized":"2024-04-18T22:22:33.241714+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:31:34.522247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127706+0000","last_change":"2024-04-18T22:22:32.144946+0000","last_active":"2024-04-18T22:22:33.127706+0000","last_peered":"2024-04-18T22:22:33.127706+0000","last_clean":"2024-04-18T22:22:33.127706+0000","last_became_active":"2024-04-18T22:22:32.143032+0000","last_became_peered":"2024-04-18T22:22:32.143032+0000","last_unstale":"2024-04-18T22:22:33.127706+0000","last_undegraded":"2024-04-18T22:22:33.127706+0000","last_fullsized":"2024-04-18T22:22:33.127706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242608+0000","last_change":"2024-04-18T22:22:31.145383+0000","last_active":"2024-04-18T22:22:33.242608+0000","last_peered":"2024-04-18T22:22:33.242608+0000","last_clean":"2024-04-18T22:22:33.242608+0000","last_became_active":"2024-04-18T22:22:31.143178+0000","last_became_peered":"2024-04-18T22:22:31.143178+0000","last_unstale":"2024-04-18T22:22:33.242608+0000","last_undegraded":"2024-04-18T22:22:33.242608+0000","last_fullsized":"2024-04-18T22:22:33.242608+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128208+0000","last_change":"2024-04-18T22:22:32.146608+0000","last_active":"2024-04-18T22:22:33.128208+0000","last_peered":"2024-04-18T22:22:33.128208+0000","last_clean":"2024-04-18T22:22:33.128208+0000","last_became_active":"2024-04-18T22:22:32.145761+0000","last_became_peered":"2024-04-18T22:22:32.145761+0000","last_unstale":"2024-04-18T22:22:33.128208+0000","last_undegraded":"2024-04-18T22:22:33.128208+0000","last_fullsized":"2024-04-18T22:22:33.128208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:48.900684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242107+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:22:33.242107+0000","last_peered":"2024-04-18T22:22:33.242107+0000","last_clean":"2024-04-18T22:22:33.242107+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:22:33.242107+0000","last_undegraded":"2024-04-18T22:22:33.242107+0000","last_fullsized":"2024-04-18T22:22:33.242107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243862+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:22:33.243862+0000","last_peered":"2024-04-18T22:22:33.243862+0000","last_clean":"2024-04-18T22:22:33.243862+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:22:33.243862+0000","last_undegraded":"2024-04-18T22:22:33.243862+0000","last_fullsized":"2024-04-18T22:22:33.243862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126186+0000","last_change":"2024-04-18T22:22:31.136552+0000","last_active":"2024-04-18T22:22:33.126186+0000","last_peered":"2024-04-18T22:22:33.126186+0000","last_clean":"2024-04-18T22:22:33.126186+0000","last_became_active":"2024-04-18T22:22:31.134824+0000","last_became_peered":"2024-04-18T22:22:31.134824+0000","last_unstale":"2024-04-18T22:22:33.126186+0000","last_undegraded":"2024-04-18T22:22:33.126186+0000","last_fullsized":"2024-04-18T22:22:33.126186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:07:17.380852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243033+0000","last_change":"2024-04-18T22:22:32.145785+0000","last_active":"2024-04-18T22:22:33.243033+0000","last_peered":"2024-04-18T22:22:33.243033+0000","last_clean":"2024-04-18T22:22:33.243033+0000","last_became_active":"2024-04-18T22:22:32.145166+0000","last_became_peered":"2024-04-18T22:22:32.145166+0000","last_unstale":"2024-04-18T22:22:33.243033+0000","last_undegraded":"2024-04-18T22:22:33.243033+0000","last_fullsized":"2024-04-18T22:22:33.243033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:46:33.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240502+0000","last_change":"2024-04-18T22:22:31.144229+0000","last_active":"2024-04-18T22:22:33.240502+0000","last_peered":"2024-04-18T22:22:33.240502+0000","last_clean":"2024-04-18T22:22:33.240502+0000","last_became_active":"2024-04-18T22:22:31.143597+0000","last_became_peered":"2024-04-18T22:22:31.143597+0000","last_unstale":"2024-04-18T22:22:33.240502+0000","last_undegraded":"2024-04-18T22:22:33.240502+0000","last_fullsized":"2024-04-18T22:22:33.240502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242009+0000","last_change":"2024-04-18T22:22:31.144343+0000","last_active":"2024-04-18T22:22:33.242009+0000","last_peered":"2024-04-18T22:22:33.242009+0000","last_clean":"2024-04-18T22:22:33.242009+0000","last_became_active":"2024-04-18T22:22:31.143325+0000","last_became_peered":"2024-04-18T22:22:31.143325+0000","last_unstale":"2024-04-18T22:22:33.242009+0000","last_undegraded":"2024-04-18T22:22:33.242009+0000","last_fullsized":"2024-04-18T22:22:33.242009+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125867+0000","last_change":"2024-04-18T22:22:32.137726+0000","last_active":"2024-04-18T22:22:33.125867+0000","last_peered":"2024-04-18T22:22:33.125867+0000","last_clean":"2024-04-18T22:22:33.125867+0000","last_became_active":"2024-04-18T22:22:32.135216+0000","last_became_peered":"2024-04-18T22:22:32.135216+0000","last_unstale":"2024-04-18T22:22:33.125867+0000","last_undegraded":"2024-04-18T22:22:33.125867+0000","last_fullsized":"2024-04-18T22:22:33.125867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:01:43.429019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128180+0000","last_change":"2024-04-18T22:22:31.131784+0000","last_active":"2024-04-18T22:22:33.128180+0000","last_peered":"2024-04-18T22:22:33.128180+0000","last_clean":"2024-04-18T22:22:33.128180+0000","last_became_active":"2024-04-18T22:22:31.131002+0000","last_became_peered":"2024-04-18T22:22:31.131002+0000","last_unstale":"2024-04-18T22:22:33.128180+0000","last_undegraded":"2024-04-18T22:22:33.128180+0000","last_fullsized":"2024-04-18T22:22:33.128180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125897+0000","last_change":"2024-04-18T22:22:32.145000+0000","last_active":"2024-04-18T22:22:33.125897+0000","last_peered":"2024-04-18T22:22:33.125897+0000","last_clean":"2024-04-18T22:22:33.125897+0000","last_became_active":"2024-04-18T22:22:32.143522+0000","last_became_peered":"2024-04-18T22:22:32.143522+0000","last_unstale":"2024-04-18T22:22:33.125897+0000","last_undegraded":"2024-04-18T22:22:33.125897+0000","last_fullsized":"2024-04-18T22:22:33.125897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:26.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.144234+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.143532+0000","last_became_peered":"2024-04-18T22:22:31.143532+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:41.528679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241436+0000","last_change":"2024-04-18T22:22:32.135717+0000","last_active":"2024-04-18T22:22:33.241436+0000","last_peered":"2024-04-18T22:22:33.241436+0000","last_clean":"2024-04-18T22:22:33.241436+0000","last_became_active":"2024-04-18T22:22:32.134770+0000","last_became_peered":"2024-04-18T22:22:32.134770+0000","last_unstale":"2024-04-18T22:22:33.241436+0000","last_undegraded":"2024-04-18T22:22:33.241436+0000","last_fullsized":"2024-04-18T22:22:33.241436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242840+0000","last_change":"2024-04-18T22:22:31.147439+0000","last_active":"2024-04-18T22:22:33.242840+0000","last_peered":"2024-04-18T22:22:33.242840+0000","last_clean":"2024-04-18T22:22:33.242840+0000","last_became_active":"2024-04-18T22:22:31.146264+0000","last_became_peered":"2024-04-18T22:22:31.146264+0000","last_unstale":"2024-04-18T22:22:33.242840+0000","last_undegraded":"2024-04-18T22:22:33.242840+0000","last_fullsized":"2024-04-18T22:22:33.242840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242813+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:22:33.242813+0000","last_peered":"2024-04-18T22:22:33.242813+0000","last_clean":"2024-04-18T22:22:33.242813+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:22:33.242813+0000","last_undegraded":"2024-04-18T22:22:33.242813+0000","last_fullsized":"2024-04-18T22:22:33.242813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242737+0000","last_change":"2024-04-18T22:22:31.140451+0000","last_active":"2024-04-18T22:22:33.242737+0000","last_peered":"2024-04-18T22:22:33.242737+0000","last_clean":"2024-04-18T22:22:33.242737+0000","last_became_active":"2024-04-18T22:22:31.139655+0000","last_became_peered":"2024-04-18T22:22:31.139655+0000","last_unstale":"2024-04-18T22:22:33.242737+0000","last_undegraded":"2024-04-18T22:22:33.242737+0000","last_fullsized":"2024-04-18T22:22:33.242737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:46:14.934334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242161+0000","last_change":"2024-04-18T22:22:32.145869+0000","last_active":"2024-04-18T22:22:33.242161+0000","last_peered":"2024-04-18T22:22:33.242161+0000","last_clean":"2024-04-18T22:22:33.242161+0000","last_became_active":"2024-04-18T22:22:32.144117+0000","last_became_peered":"2024-04-18T22:22:32.144117+0000","last_unstale":"2024-04-18T22:22:33.242161+0000","last_undegraded":"2024-04-18T22:22:33.242161+0000","last_fullsized":"2024-04-18T22:22:33.242161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:39.186356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242361+0000","last_change":"2024-04-18T22:22:31.143197+0000","last_active":"2024-04-18T22:22:33.242361+0000","last_peered":"2024-04-18T22:22:33.242361+0000","last_clean":"2024-04-18T22:22:33.242361+0000","last_became_active":"2024-04-18T22:22:31.142393+0000","last_became_peered":"2024-04-18T22:22:31.142393+0000","last_unstale":"2024-04-18T22:22:33.242361+0000","last_undegraded":"2024-04-18T22:22:33.242361+0000","last_fullsized":"2024-04-18T22:22:33.242361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243118+0000","last_change":"2024-04-18T22:22:32.147396+0000","last_active":"2024-04-18T22:22:33.243118+0000","last_peered":"2024-04-18T22:22:33.243118+0000","last_clean":"2024-04-18T22:22:33.243118+0000","last_became_active":"2024-04-18T22:22:32.146327+0000","last_became_peered":"2024-04-18T22:22:32.146327+0000","last_unstale":"2024-04-18T22:22:33.243118+0000","last_undegraded":"2024-04-18T22:22:33.243118+0000","last_fullsized":"2024-04-18T22:22:33.243118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:16:31.884436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126713+0000","last_change":"2024-04-18T22:22:31.135058+0000","last_active":"2024-04-18T22:22:33.126713+0000","last_peered":"2024-04-18T22:22:33.126713+0000","last_clean":"2024-04-18T22:22:33.126713+0000","last_became_active":"2024-04-18T22:22:31.134213+0000","last_became_peered":"2024-04-18T22:22:31.134213+0000","last_unstale":"2024-04-18T22:22:33.126713+0000","last_undegraded":"2024-04-18T22:22:33.126713+0000","last_fullsized":"2024-04-18T22:22:33.126713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:35.837015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241581+0000","last_change":"2024-04-18T22:22:32.144956+0000","last_active":"2024-04-18T22:22:33.241581+0000","last_peered":"2024-04-18T22:22:33.241581+0000","last_clean":"2024-04-18T22:22:33.241581+0000","last_became_active":"2024-04-18T22:22:32.144171+0000","last_became_peered":"2024-04-18T22:22:32.144171+0000","last_unstale":"2024-04-18T22:22:33.241581+0000","last_undegraded":"2024-04-18T22:22:33.241581+0000","last_fullsized":"2024-04-18T22:22:33.241581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126257+0000","last_change":"2024-04-18T22:22:31.129222+0000","last_active":"2024-04-18T22:22:33.126257+0000","last_peered":"2024-04-18T22:22:33.126257+0000","last_clean":"2024-04-18T22:22:33.126257+0000","last_became_active":"2024-04-18T22:22:31.128618+0000","last_became_peered":"2024-04-18T22:22:31.128618+0000","last_unstale":"2024-04-18T22:22:33.126257+0000","last_undegraded":"2024-04-18T22:22:33.126257+0000","last_fullsized":"2024-04-18T22:22:33.126257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:58:49.687595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243356+0000","last_change":"2024-04-18T22:22:32.150911+0000","last_active":"2024-04-18T22:22:33.243356+0000","last_peered":"2024-04-18T22:22:33.243356+0000","last_clean":"2024-04-18T22:22:33.243356+0000","last_became_active":"2024-04-18T22:22:32.149188+0000","last_became_peered":"2024-04-18T22:22:32.149188+0000","last_unstale":"2024-04-18T22:22:33.243356+0000","last_undegraded":"2024-04-18T22:22:33.243356+0000","last_fullsized":"2024-04-18T22:22:33.243356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:57.165125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":222,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242605+0000","last_change":"2024-04-18T22:22:21.927447+0000","last_active":"2024-04-18T22:22:33.242605+0000","last_peered":"2024-04-18T22:22:33.242605+0000","last_clean":"2024-04-18T22:22:33.242605+0000","last_became_active":"2024-04-18T22:22:21.926785+0000","last_became_peered":"2024-04-18T22:22:21.926785+0000","last_unstale":"2024-04-18T22:22:33.242605+0000","last_undegraded":"2024-04-18T22:22:33.242605+0000","last_fullsized":"2024-04-18T22:22:33.242605+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:36:50.586559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242816+0000","last_change":"2024-04-18T22:22:27.090489+0000","last_active":"2024-04-18T22:22:33.242816+0000","last_peered":"2024-04-18T22:22:33.242816+0000","last_clean":"2024-04-18T22:22:33.242816+0000","last_became_active":"2024-04-18T22:22:27.089797+0000","last_became_peered":"2024-04-18T22:22:27.089797+0000","last_unstale":"2024-04-18T22:22:33.242816+0000","last_undegraded":"2024-04-18T22:22:33.242816+0000","last_fullsized":"2024-04-18T22:22:33.242816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:16.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242531+0000","last_change":"2024-04-18T22:22:31.128030+0000","last_active":"2024-04-18T22:22:33.242531+0000","last_peered":"2024-04-18T22:22:33.242531+0000","last_clean":"2024-04-18T22:22:33.242531+0000","last_became_active":"2024-04-18T22:22:31.126939+0000","last_became_peered":"2024-04-18T22:22:31.126939+0000","last_unstale":"2024-04-18T22:22:33.242531+0000","last_undegraded":"2024-04-18T22:22:33.242531+0000","last_fullsized":"2024-04-18T22:22:33.242531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243540+0000","last_change":"2024-04-18T22:22:32.153417+0000","last_active":"2024-04-18T22:22:33.243540+0000","last_peered":"2024-04-18T22:22:33.243540+0000","last_clean":"2024-04-18T22:22:33.243540+0000","last_became_active":"2024-04-18T22:22:32.152653+0000","last_became_peered":"2024-04-18T22:22:32.152653+0000","last_unstale":"2024-04-18T22:22:33.243540+0000","last_undegraded":"2024-04-18T22:22:33.243540+0000","last_fullsized":"2024-04-18T22:22:33.243540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:49:12.269993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242307+0000","last_change":"2024-04-18T22:22:27.089211+0000","last_active":"2024-04-18T22:22:33.242307+0000","last_peered":"2024-04-18T22:22:33.242307+0000","last_clean":"2024-04-18T22:22:33.242307+0000","last_became_active":"2024-04-18T22:22:27.088702+0000","last_became_peered":"2024-04-18T22:22:27.088702+0000","last_unstale":"2024-04-18T22:22:33.242307+0000","last_undegraded":"2024-04-18T22:22:33.242307+0000","last_fullsized":"2024-04-18T22:22:33.242307+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:59:37.183874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242019+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:22:33.242019+0000","last_peered":"2024-04-18T22:22:33.242019+0000","last_clean":"2024-04-18T22:22:33.242019+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:22:33.242019+0000","last_undegraded":"2024-04-18T22:22:33.242019+0000","last_fullsized":"2024-04-18T22:22:33.242019+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127898+0000","last_change":"2024-04-18T22:22:32.147651+0000","last_active":"2024-04-18T22:22:33.127898+0000","last_peered":"2024-04-18T22:22:33.127898+0000","last_clean":"2024-04-18T22:22:33.127898+0000","last_became_active":"2024-04-18T22:22:32.143774+0000","last_became_peered":"2024-04-18T22:22:32.143774+0000","last_unstale":"2024-04-18T22:22:33.127898+0000","last_undegraded":"2024-04-18T22:22:33.127898+0000","last_fullsized":"2024-04-18T22:22:33.127898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242167+0000","last_change":"2024-04-18T22:22:27.090391+0000","last_active":"2024-04-18T22:22:33.242167+0000","last_peered":"2024-04-18T22:22:33.242167+0000","last_clean":"2024-04-18T22:22:33.242167+0000","last_became_active":"2024-04-18T22:22:27.089444+0000","last_became_peered":"2024-04-18T22:22:27.089444+0000","last_unstale":"2024-04-18T22:22:33.242167+0000","last_undegraded":"2024-04-18T22:22:33.242167+0000","last_fullsized":"2024-04-18T22:22:33.242167+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:51:23.828198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126710+0000","last_change":"2024-04-18T22:22:31.132958+0000","last_active":"2024-04-18T22:22:33.126710+0000","last_peered":"2024-04-18T22:22:33.126710+0000","last_clean":"2024-04-18T22:22:33.126710+0000","last_became_active":"2024-04-18T22:22:31.132186+0000","last_became_peered":"2024-04-18T22:22:31.132186+0000","last_unstale":"2024-04-18T22:22:33.126710+0000","last_undegraded":"2024-04-18T22:22:33.126710+0000","last_fullsized":"2024-04-18T22:22:33.126710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.665986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243413+0000","last_change":"2024-04-18T22:22:32.150208+0000","last_active":"2024-04-18T22:22:33.243413+0000","last_peered":"2024-04-18T22:22:33.243413+0000","last_clean":"2024-04-18T22:22:33.243413+0000","last_became_active":"2024-04-18T22:22:32.148155+0000","last_became_peered":"2024-04-18T22:22:32.148155+0000","last_unstale":"2024-04-18T22:22:33.243413+0000","last_undegraded":"2024-04-18T22:22:33.243413+0000","last_fullsized":"2024-04-18T22:22:33.243413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241760+0000","last_change":"2024-04-18T22:22:27.090591+0000","last_active":"2024-04-18T22:22:33.241760+0000","last_peered":"2024-04-18T22:22:33.241760+0000","last_clean":"2024-04-18T22:22:33.241760+0000","last_became_active":"2024-04-18T22:22:27.090131+0000","last_became_peered":"2024-04-18T22:22:27.090131+0000","last_unstale":"2024-04-18T22:22:33.241760+0000","last_undegraded":"2024-04-18T22:22:33.241760+0000","last_fullsized":"2024-04-18T22:22:33.241760+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:10.668000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242877+0000","last_change":"2024-04-18T22:22:31.146898+0000","last_active":"2024-04-18T22:22:33.242877+0000","last_peered":"2024-04-18T22:22:33.242877+0000","last_clean":"2024-04-18T22:22:33.242877+0000","last_became_active":"2024-04-18T22:22:31.146071+0000","last_became_peered":"2024-04-18T22:22:31.146071+0000","last_unstale":"2024-04-18T22:22:33.242877+0000","last_undegraded":"2024-04-18T22:22:33.242877+0000","last_fullsized":"2024-04-18T22:22:33.242877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243542+0000","last_change":"2024-04-18T22:22:32.147890+0000","last_active":"2024-04-18T22:22:33.243542+0000","last_peered":"2024-04-18T22:22:33.243542+0000","last_clean":"2024-04-18T22:22:33.243542+0000","last_became_active":"2024-04-18T22:22:32.146799+0000","last_became_peered":"2024-04-18T22:22:32.146799+0000","last_unstale":"2024-04-18T22:22:33.243542+0000","last_undegraded":"2024-04-18T22:22:33.243542+0000","last_fullsized":"2024-04-18T22:22:33.243542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243666+0000","last_change":"2024-04-18T22:22:31.146681+0000","last_active":"2024-04-18T22:22:33.243666+0000","last_peered":"2024-04-18T22:22:33.243666+0000","last_clean":"2024-04-18T22:22:33.243666+0000","last_became_active":"2024-04-18T22:22:31.145764+0000","last_became_peered":"2024-04-18T22:22:31.145764+0000","last_unstale":"2024-04-18T22:22:33.243666+0000","last_undegraded":"2024-04-18T22:22:33.243666+0000","last_fullsized":"2024-04-18T22:22:33.243666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:33.679846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243510+0000","last_change":"2024-04-18T22:22:27.090296+0000","last_active":"2024-04-18T22:22:33.243510+0000","last_peered":"2024-04-18T22:22:33.243510+0000","last_clean":"2024-04-18T22:22:33.243510+0000","last_became_active":"2024-04-18T22:22:27.089619+0000","last_became_peered":"2024-04-18T22:22:27.089619+0000","last_unstale":"2024-04-18T22:22:33.243510+0000","last_undegraded":"2024-04-18T22:22:33.243510+0000","last_fullsized":"2024-04-18T22:22:33.243510+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:41:10.321620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127522+0000","last_change":"2024-04-18T22:22:32.127971+0000","last_active":"2024-04-18T22:22:33.127522+0000","last_peered":"2024-04-18T22:22:33.127522+0000","last_clean":"2024-04-18T22:22:33.127522+0000","last_became_active":"2024-04-18T22:22:32.126518+0000","last_became_peered":"2024-04-18T22:22:32.126518+0000","last_unstale":"2024-04-18T22:22:33.127522+0000","last_undegraded":"2024-04-18T22:22:33.127522+0000","last_fullsized":"2024-04-18T22:22:33.127522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243415+0000","last_change":"2024-04-18T22:22:31.127787+0000","last_active":"2024-04-18T22:22:33.243415+0000","last_peered":"2024-04-18T22:22:33.243415+0000","last_clean":"2024-04-18T22:22:33.243415+0000","last_became_active":"2024-04-18T22:22:31.127173+0000","last_became_peered":"2024-04-18T22:22:31.127173+0000","last_unstale":"2024-04-18T22:22:33.243415+0000","last_undegraded":"2024-04-18T22:22:33.243415+0000","last_fullsized":"2024-04-18T22:22:33.243415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:50:27.921073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128505+0000","last_change":"2024-04-18T22:22:27.089438+0000","last_active":"2024-04-18T22:22:33.128505+0000","last_peered":"2024-04-18T22:22:33.128505+0000","last_clean":"2024-04-18T22:22:33.128505+0000","last_became_active":"2024-04-18T22:22:27.088422+0000","last_became_peered":"2024-04-18T22:22:27.088422+0000","last_unstale":"2024-04-18T22:22:33.128505+0000","last_undegraded":"2024-04-18T22:22:33.128505+0000","last_fullsized":"2024-04-18T22:22:33.128505+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:24.222387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126910+0000","last_change":"2024-04-18T22:22:32.136161+0000","last_active":"2024-04-18T22:22:33.126910+0000","last_peered":"2024-04-18T22:22:33.126910+0000","last_clean":"2024-04-18T22:22:33.126910+0000","last_became_active":"2024-04-18T22:22:32.135253+0000","last_became_peered":"2024-04-18T22:22:32.135253+0000","last_unstale":"2024-04-18T22:22:33.126910+0000","last_undegraded":"2024-04-18T22:22:33.126910+0000","last_fullsized":"2024-04-18T22:22:33.126910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:47:14.010377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244171+0000","last_change":"2024-04-18T22:22:31.136319+0000","last_active":"2024-04-18T22:22:33.244171+0000","last_peered":"2024-04-18T22:22:33.244171+0000","last_clean":"2024-04-18T22:22:33.244171+0000","last_became_active":"2024-04-18T22:22:31.135689+0000","last_became_peered":"2024-04-18T22:22:31.135689+0000","last_unstale":"2024-04-18T22:22:33.244171+0000","last_undegraded":"2024-04-18T22:22:33.244171+0000","last_fullsized":"2024-04-18T22:22:33.244171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:56:48.796901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.244003+0000","last_change":"2024-04-18T22:22:27.090559+0000","last_active":"2024-04-18T22:22:33.244003+0000","last_peered":"2024-04-18T22:22:33.244003+0000","last_clean":"2024-04-18T22:22:33.244003+0000","last_became_active":"2024-04-18T22:22:27.089674+0000","last_became_peered":"2024-04-18T22:22:27.089674+0000","last_unstale":"2024-04-18T22:22:33.244003+0000","last_undegraded":"2024-04-18T22:22:33.244003+0000","last_fullsized":"2024-04-18T22:22:33.244003+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:59:02.127503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242936+0000","last_change":"2024-04-18T22:22:32.144432+0000","last_active":"2024-04-18T22:22:33.242936+0000","last_peered":"2024-04-18T22:22:33.242936+0000","last_clean":"2024-04-18T22:22:33.242936+0000","last_became_active":"2024-04-18T22:22:32.142947+0000","last_became_peered":"2024-04-18T22:22:32.142947+0000","last_unstale":"2024-04-18T22:22:33.242936+0000","last_undegraded":"2024-04-18T22:22:33.242936+0000","last_fullsized":"2024-04-18T22:22:33.242936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:41:38.451344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:34.119061+0000","last_change":"2024-04-18T22:22:31.143202+0000","last_active":"2024-04-18T22:22:34.119061+0000","last_peered":"2024-04-18T22:22:34.119061+0000","last_clean":"2024-04-18T22:22:34.119061+0000","last_became_active":"2024-04-18T22:22:31.142465+0000","last_became_peered":"2024-04-18T22:22:31.142465+0000","last_unstale":"2024-04-18T22:22:34.119061+0000","last_undegraded":"2024-04-18T22:22:34.119061+0000","last_fullsized":"2024-04-18T22:22:34.119061+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:05:35.523788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241961+0000","last_change":"2024-04-18T22:22:27.089434+0000","last_active":"2024-04-18T22:22:33.241961+0000","last_peered":"2024-04-18T22:22:33.241961+0000","last_clean":"2024-04-18T22:22:33.241961+0000","last_became_active":"2024-04-18T22:22:27.088778+0000","last_became_peered":"2024-04-18T22:22:27.088778+0000","last_unstale":"2024-04-18T22:22:33.241961+0000","last_undegraded":"2024-04-18T22:22:33.241961+0000","last_fullsized":"2024-04-18T22:22:33.241961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:54:38.921512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.125912+0000","last_change":"2024-04-18T22:22:32.132968+0000","last_active":"2024-04-18T22:22:33.125912+0000","last_peered":"2024-04-18T22:22:33.125912+0000","last_clean":"2024-04-18T22:22:33.125912+0000","last_became_active":"2024-04-18T22:22:32.131596+0000","last_became_peered":"2024-04-18T22:22:32.131596+0000","last_unstale":"2024-04-18T22:22:33.125912+0000","last_undegraded":"2024-04-18T22:22:33.125912+0000","last_fullsized":"2024-04-18T22:22:33.125912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241097+0000","last_change":"2024-04-18T22:22:32.145633+0000","last_active":"2024-04-18T22:22:33.241097+0000","last_peered":"2024-04-18T22:22:33.241097+0000","last_clean":"2024-04-18T22:22:33.241097+0000","last_became_active":"2024-04-18T22:22:32.142868+0000","last_became_peered":"2024-04-18T22:22:32.142868+0000","last_unstale":"2024-04-18T22:22:33.241097+0000","last_undegraded":"2024-04-18T22:22:33.241097+0000","last_fullsized":"2024-04-18T22:22:33.241097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242663+0000","last_change":"2024-04-18T22:22:31.144413+0000","last_active":"2024-04-18T22:22:33.242663+0000","last_peered":"2024-04-18T22:22:33.242663+0000","last_clean":"2024-04-18T22:22:33.242663+0000","last_became_active":"2024-04-18T22:22:31.143704+0000","last_became_peered":"2024-04-18T22:22:31.143704+0000","last_unstale":"2024-04-18T22:22:33.242663+0000","last_undegraded":"2024-04-18T22:22:33.242663+0000","last_fullsized":"2024-04-18T22:22:33.242663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241992+0000","last_change":"2024-04-18T22:22:32.148407+0000","last_active":"2024-04-18T22:22:33.241992+0000","last_peered":"2024-04-18T22:22:33.241992+0000","last_clean":"2024-04-18T22:22:33.241992+0000","last_became_active":"2024-04-18T22:22:32.147033+0000","last_became_peered":"2024-04-18T22:22:32.147033+0000","last_unstale":"2024-04-18T22:22:33.241992+0000","last_undegraded":"2024-04-18T22:22:33.241992+0000","last_fullsized":"2024-04-18T22:22:33.241992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:22:46.218621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128409+0000","last_change":"2024-04-18T22:22:31.133306+0000","last_active":"2024-04-18T22:22:33.128409+0000","last_peered":"2024-04-18T22:22:33.128409+0000","last_clean":"2024-04-18T22:22:33.128409+0000","last_became_active":"2024-04-18T22:22:31.132648+0000","last_became_peered":"2024-04-18T22:22:31.132648+0000","last_unstale":"2024-04-18T22:22:33.128409+0000","last_undegraded":"2024-04-18T22:22:33.128409+0000","last_fullsized":"2024-04-18T22:22:33.128409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242551+0000","last_change":"2024-04-18T22:22:31.146827+0000","last_active":"2024-04-18T22:22:33.242551+0000","last_peered":"2024-04-18T22:22:33.242551+0000","last_clean":"2024-04-18T22:22:33.242551+0000","last_became_active":"2024-04-18T22:22:31.146063+0000","last_became_peered":"2024-04-18T22:22:31.146063+0000","last_unstale":"2024-04-18T22:22:33.242551+0000","last_undegraded":"2024-04-18T22:22:33.242551+0000","last_fullsized":"2024-04-18T22:22:33.242551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:06:11.838841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241955+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:22:33.241955+0000","last_peered":"2024-04-18T22:22:33.241955+0000","last_clean":"2024-04-18T22:22:33.241955+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:22:33.241955+0000","last_undegraded":"2024-04-18T22:22:33.241955+0000","last_fullsized":"2024-04-18T22:22:33.241955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243856+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:22:33.243856+0000","last_peered":"2024-04-18T22:22:33.243856+0000","last_clean":"2024-04-18T22:22:33.243856+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:22:33.243856+0000","last_undegraded":"2024-04-18T22:22:33.243856+0000","last_fullsized":"2024-04-18T22:22:33.243856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241179+0000","last_change":"2024-04-18T22:22:32.146927+0000","last_active":"2024-04-18T22:22:33.241179+0000","last_peered":"2024-04-18T22:22:33.241179+0000","last_clean":"2024-04-18T22:22:33.241179+0000","last_became_active":"2024-04-18T22:22:32.146040+0000","last_became_peered":"2024-04-18T22:22:32.146040+0000","last_unstale":"2024-04-18T22:22:33.241179+0000","last_undegraded":"2024-04-18T22:22:33.241179+0000","last_fullsized":"2024-04-18T22:22:33.241179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:39:04.462900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128951+0000","last_change":"2024-04-18T22:22:32.139063+0000","last_active":"2024-04-18T22:22:33.128951+0000","last_peered":"2024-04-18T22:22:33.128951+0000","last_clean":"2024-04-18T22:22:33.128951+0000","last_became_active":"2024-04-18T22:22:32.138016+0000","last_became_peered":"2024-04-18T22:22:32.138016+0000","last_unstale":"2024-04-18T22:22:33.128951+0000","last_undegraded":"2024-04-18T22:22:33.128951+0000","last_fullsized":"2024-04-18T22:22:33.128951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241615+0000","last_change":"2024-04-18T22:22:31.130452+0000","last_active":"2024-04-18T22:22:33.241615+0000","last_peered":"2024-04-18T22:22:33.241615+0000","last_clean":"2024-04-18T22:22:33.241615+0000","last_became_active":"2024-04-18T22:22:31.129555+0000","last_became_peered":"2024-04-18T22:22:31.129555+0000","last_unstale":"2024-04-18T22:22:33.241615+0000","last_undegraded":"2024-04-18T22:22:33.241615+0000","last_fullsized":"2024-04-18T22:22:33.241615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243327+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:22:33.243327+0000","last_peered":"2024-04-18T22:22:33.243327+0000","last_clean":"2024-04-18T22:22:33.243327+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:22:33.243327+0000","last_undegraded":"2024-04-18T22:22:33.243327+0000","last_fullsized":"2024-04-18T22:22:33.243327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241696+0000","last_change":"2024-04-18T22:22:31.133021+0000","last_active":"2024-04-18T22:22:33.241696+0000","last_peered":"2024-04-18T22:22:33.241696+0000","last_clean":"2024-04-18T22:22:33.241696+0000","last_became_active":"2024-04-18T22:22:31.132544+0000","last_became_peered":"2024-04-18T22:22:31.132544+0000","last_unstale":"2024-04-18T22:22:33.241696+0000","last_undegraded":"2024-04-18T22:22:33.241696+0000","last_fullsized":"2024-04-18T22:22:33.241696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:51.116981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127774+0000","last_change":"2024-04-18T22:22:32.138696+0000","last_active":"2024-04-18T22:22:33.127774+0000","last_peered":"2024-04-18T22:22:33.127774+0000","last_clean":"2024-04-18T22:22:33.127774+0000","last_became_active":"2024-04-18T22:22:32.137900+0000","last_became_peered":"2024-04-18T22:22:32.137900+0000","last_unstale":"2024-04-18T22:22:33.127774+0000","last_undegraded":"2024-04-18T22:22:33.127774+0000","last_fullsized":"2024-04-18T22:22:33.127774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:45:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.126866+0000","last_change":"2024-04-18T22:22:31.133358+0000","last_active":"2024-04-18T22:22:33.126866+0000","last_peered":"2024-04-18T22:22:33.126866+0000","last_clean":"2024-04-18T22:22:33.126866+0000","last_became_active":"2024-04-18T22:22:31.132642+0000","last_became_peered":"2024-04-18T22:22:31.132642+0000","last_unstale":"2024-04-18T22:22:33.126866+0000","last_undegraded":"2024-04-18T22:22:33.126866+0000","last_fullsized":"2024-04-18T22:22:33.126866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127236+0000","last_change":"2024-04-18T22:22:32.146126+0000","last_active":"2024-04-18T22:22:33.127236+0000","last_peered":"2024-04-18T22:22:33.127236+0000","last_clean":"2024-04-18T22:22:33.127236+0000","last_became_active":"2024-04-18T22:22:32.144083+0000","last_became_peered":"2024-04-18T22:22:32.144083+0000","last_unstale":"2024-04-18T22:22:33.127236+0000","last_undegraded":"2024-04-18T22:22:33.127236+0000","last_fullsized":"2024-04-18T22:22:33.127236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:09:13.769504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127648+0000","last_change":"2024-04-18T22:22:31.136636+0000","last_active":"2024-04-18T22:22:33.127648+0000","last_peered":"2024-04-18T22:22:33.127648+0000","last_clean":"2024-04-18T22:22:33.127648+0000","last_became_active":"2024-04-18T22:22:31.135039+0000","last_became_peered":"2024-04-18T22:22:31.135039+0000","last_unstale":"2024-04-18T22:22:33.127648+0000","last_undegraded":"2024-04-18T22:22:33.127648+0000","last_fullsized":"2024-04-18T22:22:33.127648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.293851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.240793+0000","last_change":"2024-04-18T22:22:32.134794+0000","last_active":"2024-04-18T22:22:33.240793+0000","last_peered":"2024-04-18T22:22:33.240793+0000","last_clean":"2024-04-18T22:22:33.240793+0000","last_became_active":"2024-04-18T22:22:32.133858+0000","last_became_peered":"2024-04-18T22:22:32.133858+0000","last_unstale":"2024-04-18T22:22:33.240793+0000","last_undegraded":"2024-04-18T22:22:33.240793+0000","last_fullsized":"2024-04-18T22:22:33.240793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:41:45.274833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242444+0000","last_change":"2024-04-18T22:22:31.147456+0000","last_active":"2024-04-18T22:22:33.242444+0000","last_peered":"2024-04-18T22:22:33.242444+0000","last_clean":"2024-04-18T22:22:33.242444+0000","last_became_active":"2024-04-18T22:22:31.145624+0000","last_became_peered":"2024-04-18T22:22:31.145624+0000","last_unstale":"2024-04-18T22:22:33.242444+0000","last_undegraded":"2024-04-18T22:22:33.242444+0000","last_fullsized":"2024-04-18T22:22:33.242444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.241579+0000","last_change":"2024-04-18T22:22:32.144038+0000","last_active":"2024-04-18T22:22:33.241579+0000","last_peered":"2024-04-18T22:22:33.241579+0000","last_clean":"2024-04-18T22:22:33.241579+0000","last_became_active":"2024-04-18T22:22:32.143184+0000","last_became_peered":"2024-04-18T22:22:32.143184+0000","last_unstale":"2024-04-18T22:22:33.241579+0000","last_undegraded":"2024-04-18T22:22:33.241579+0000","last_fullsized":"2024-04-18T22:22:33.241579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:15:02.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.128174+0000","last_change":"2024-04-18T22:22:31.138492+0000","last_active":"2024-04-18T22:22:33.128174+0000","last_peered":"2024-04-18T22:22:33.128174+0000","last_clean":"2024-04-18T22:22:33.128174+0000","last_became_active":"2024-04-18T22:22:31.134646+0000","last_became_peered":"2024-04-18T22:22:31.134646+0000","last_unstale":"2024-04-18T22:22:33.128174+0000","last_undegraded":"2024-04-18T22:22:33.128174+0000","last_fullsized":"2024-04-18T22:22:33.128174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:00:19.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.127146+0000","last_change":"2024-04-18T22:22:32.137473+0000","last_active":"2024-04-18T22:22:33.127146+0000","last_peered":"2024-04-18T22:22:33.127146+0000","last_clean":"2024-04-18T22:22:33.127146+0000","last_became_active":"2024-04-18T22:22:32.136025+0000","last_became_peered":"2024-04-18T22:22:32.136025+0000","last_unstale":"2024-04-18T22:22:33.127146+0000","last_undegraded":"2024-04-18T22:22:33.127146+0000","last_fullsized":"2024-04-18T22:22:33.127146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:31.069855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243824+0000","last_change":"2024-04-18T22:22:31.130827+0000","last_active":"2024-04-18T22:22:33.243824+0000","last_peered":"2024-04-18T22:22:33.243824+0000","last_clean":"2024-04-18T22:22:33.243824+0000","last_became_active":"2024-04-18T22:22:31.129728+0000","last_became_peered":"2024-04-18T22:22:31.129728+0000","last_unstale":"2024-04-18T22:22:33.243824+0000","last_undegraded":"2024-04-18T22:22:33.243824+0000","last_fullsized":"2024-04-18T22:22:33.243824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:19.112080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.242152+0000","last_change":"2024-04-18T22:22:32.144325+0000","last_active":"2024-04-18T22:22:33.242152+0000","last_peered":"2024-04-18T22:22:33.242152+0000","last_clean":"2024-04-18T22:22:33.242152+0000","last_became_active":"2024-04-18T22:22:32.143495+0000","last_became_peered":"2024-04-18T22:22:32.143495+0000","last_unstale":"2024-04-18T22:22:33.242152+0000","last_undegraded":"2024-04-18T22:22:33.242152+0000","last_fullsized":"2024-04-18T22:22:33.242152+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:25:14.141330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-18T22:22:33.243703+0000","last_change":"2024-04-18T22:22:31.142640+0000","last_active":"2024-04-18T22:22:33.243703+0000","last_peered":"2024-04-18T22:22:33.243703+0000","last_clean":"2024-04-18T22:22:33.243703+0000","last_became_active":"2024-04-18T22:22:31.141819+0000","last_became_peered":"2024-04-18T22:22:31.141819+0000","last_unstale":"2024-04-18T22:22:33.243703+0000","last_undegraded":"2024-04-18T22:22:33.243703+0000","last_fullsized":"2024-04-18T22:22:33.243703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:22:32.842500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738378,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":245760,"data_stored":647890,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":647274,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":55728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:22:52.609 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-18T22:22:52.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-18T22:22:52.954 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:fsid a1b9e9bc-8f1f-4426-9796-fae392070eec 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-18T22:22:10.044535+0000 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-04-18T22:22:33.115398+0000 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 4 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client jewel 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release squid 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2024-04-18T22:22:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 11 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 2.99 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool stripe_width 0 application rbd read_balance_score 1.50 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:pool 3 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.13 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:pool 4 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.13 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 3 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 8 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/4267325794,v1:172.21.15.86:6801/4267325794] [v2:172.21.15.86:6802/4267325794,v1:172.21.15.86:6803/4267325794] exists,up 57238913-8d4f-4ea8-a1a5-92ce7aceaf4a 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 8 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/3490435725,v1:172.21.15.96:6801/3490435725] [v2:172.21.15.96:6802/3490435725,v1:172.21.15.96:6803/3490435725] exists,up 68bfe465-447e-4083-a8d5-e6f996b2bb2c 2024-04-18T22:22:52.956 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 8 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2337964302,v1:172.21.15.196:6801/2337964302] [v2:172.21.15.196:6802/2337964302,v1:172.21.15.196:6803/2337964302] exists,up 32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e 2024-04-18T22:22:52.972 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/4267325794,v1:172.21.15.86:6801/4267325794] [v2:172.21.15.86:6802/4267325794,v1:172.21.15.86:6803/4267325794] exists,up 57238913-8d4f-4ea8-a1a5-92ce7aceaf4a', 'osd.1 up in weight 1 up_from 8 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/3490435725,v1:172.21.15.96:6801/3490435725] [v2:172.21.15.96:6802/3490435725,v1:172.21.15.96:6803/3490435725] exists,up 68bfe465-447e-4083-a8d5-e6f996b2bb2c', 'osd.2 up in weight 1 up_from 8 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2337964302,v1:172.21.15.196:6801/2337964302] [v2:172.21.15.196:6802/2337964302,v1:172.21.15.196:6803/2337964302] exists,up 32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e'] 2024-04-18T22:22:52.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mon.a", 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster": "ceph", 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "admin_socket_mode": "", 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-18T22:22:53.091 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_client_required": "cephx, none", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_cluster_required": "cephx", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_debug": "true", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_service_required": "cephx", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_supported": "cephx", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio": "true", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_poll_ms": "250", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_reap_max": "16", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_async_discard": "false", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_block_size": "4096", 2024-04-18T22:22:53.092 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_inflight_ios": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_enable_discard": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_flock_retry": "3", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_inject_crash": "0", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring_hipri": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-18T22:22:53.093 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_type": "", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_alloc_size": "1048576", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_allocator": "hybrid", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_buffered_io": "true", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_for_zeros": "false", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_compact_log_sync": "false", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_max_log_runway": "4194304", 2024-04-18T22:22:53.094 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_max_prefetch": "1048576", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_min_flush_size": "524288", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_min_log_runway": "1048576", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_replay_recovery": "false", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_sync_write": "false", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_allocation_from_file": "true", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_allocator": "hybrid", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-18T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_create": "true", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_create": "false", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_path": "", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_size": "0", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_path": "", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_preallocate_file": "false", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_size": "107374182400", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_create": "false", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_path": "", 2024-04-18T22:22:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_size": "100663296", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs": "true", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_autotune": "true", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-18T22:22:53.097 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size": "0", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_type": "2q", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_clone_cow": "true", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-18T22:22:53.098 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_mode": "none", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_csum_type": "crc32c", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_freelist": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-18T22:22:53.099 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_small_allocations": "0", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_default_buffered_read": "true", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_default_buffered_write": "false", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-18T22:22:53.100 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fail_eio": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mount": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_umount": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-18T22:22:53.101 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_ignore_data_csum": "false", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_op_age": "5.000000", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_alloc_size": "0", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size": "0", 2024-04-18T22:22:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_deferred_txc": "32", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size": "0", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_nid_prealloc": "1024", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_retry_disk_reads": "3", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_cf": "true", 2024-04-18T22:22:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_mem": "512", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-18T22:22:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_tracing": "false", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_zero_block_detection": "false", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-18T22:22:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_cluster_require_signatures": "false", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_cluster_require_version": "2", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_require_signatures": "false", 2024-04-18T22:22:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_require_version": "2", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_service_require_signatures": "false", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_service_require_version": "2", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_sign_messages": "true", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "chdir": "", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_acl_type": "", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_asio_thread_count": "2", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_cache_mid": "0.750000", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_cache_size": "16384", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_caps_release_delay": "5", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_check_pool_perm": "true", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_force_sync_read": "false", 2024-04-18T22:22:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_getattr_caps": "false", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_inject_tick_delay": "0", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_die_on_failed_remount": "false", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_dirsize_rbytes": "true", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_force_lazyio": "false", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_fs": "", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_inject_release_failure": "false", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_max_inline_size": "4096", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mds_namespace": "", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_metadata": "", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_gid": "-1", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_timeout": "300", 2024-04-18T22:22:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_uid": "-1", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mountpoint": "/", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_notify_timeout": "10", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc": "true", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_dirty": "104857600", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_objects": "1000", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_size": "209715200", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_target_dirty": "8388608", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_permissions": "true", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_quota": "true", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_quota_df": "true", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_max_bytes": "0", 2024-04-18T22:22:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_max_periods": "4", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_min": "131072", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_reconnect_stale": "false", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_shutdown_timeout": "30", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_snapdir": ".snap", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_tick_interval": "1", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_trace": "", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_try_dentry_invalidate": "false", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_use_faked_inos": "false", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_use_random_mds": "false", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog": "false", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog_port": "12201", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_monitors": "default=true", 2024-04-18T22:22:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog": "false", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog_level": "info", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "-", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_network": "", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_network_interface": "", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_isal": "false", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_level": "5", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_winsize": "-15", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zstd_level": "1", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_alien_op_num_threads": "6", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-18T22:22:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_seastar_cpu_cores": "", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location": "", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location_hook": "", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location_hook_timeout": "10", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "daemonize": "false", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "daos_pool": "tank", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_alienstore": "0/5", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_allow_any_pool_priority": "false", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asok": "1/5", 2024-04-18T22:22:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asok_assert_abort": "false", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asserts_on_shutdown": "true", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_auth": "20/20", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bdev": "1/3", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bluefs": "1/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bluestore": "1/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_buffer": "0/1", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_ceph_exporter": "1/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cephfs_mirror": "0/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cephsqlite": "0/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_client": "0/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_compressor": "1/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_context": "0/1", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crimson_interrupt": "0/5", 2024-04-18T22:22:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crush": "1/1", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crypto": "1/5", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cyanstore": "0/5", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_deliberately_leak_memory": "false", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_disable_randomized_ping": "false", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_dpdk": "1/5", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_eventtrace": "1/5", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_filer": "0/1", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_filestore": "1/3", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_finisher": "1/1", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_fuse": "1/5", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_heartbeat_testing_span": "0", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_heartbeatmap": "1/5", 2024-04-18T22:22:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_javaclient": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_journal": "1/3", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_journaler": "0/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_kstore": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_lockdep": "0/1", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mclock": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_balancer": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_locker": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_log": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_log_expire": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_migrator": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_quiesce": "3/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_memstore": "1/5", 2024-04-18T22:22:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mgr": "2/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mgrc": "1/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mon": "20/20", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_monc": "0/10", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_ms": "1/1", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_none": "0/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objclass": "0/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objectcacher": "0/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objecter": "0/1", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_optracker": "0/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_osd": "1/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_paxos": "20/20", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_perfcounter": "1/5", 2024-04-18T22:22:53.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_prioritycache": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rados": "0/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd": "0/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_mirror": "0/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_pwl": "0/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_replay": "0/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_refs": "0/0", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_reserver": "1/1", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_access": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_datacache": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_dbstore": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_flight": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-18T22:22:53.117 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_sync": "1/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rocksdb": "4/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_backref": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_cache": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_cleaner": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_device": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_epm": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_journal": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_lba": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_odata": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_omap": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_onode": "0/5", 2024-04-18T22:22:53.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_t": "0/5", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_tm": "0/5", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_striper": "0/1", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_test": "0/5", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_throttle": "1/1", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_timer": "0/1", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_tp": "0/5", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_trace": "1/5", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "device_failure_prediction_mode": "none", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_graylog": "false", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_journald": "false", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_stderr": "true", 2024-04-18T22:22:53.119 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_syslog": "false", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event_tracing": "false", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_addr": "0.0.0.0", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_http_port": "9926", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_prio_limit": "5", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_sort_metrics": "true", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_stats_period": "5", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "fake_statfs_for_testing": "0", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "fatal_signal_handlers": "true", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "filer_max_purge_ops": "10", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "filer_max_truncate_ops": "128", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_apply_finisher_threads": "1", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_blackhole": "false", 2024-04-18T22:22:53.120 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_btrfs_clone_range": "true", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_btrfs_snap": "true", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_caller_concurrency": "10", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_collect_device_partition_information": "true", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_commit_timeout": "600.000000", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_inject_read_err": "false", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_omap_check": "false", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_verify_split": "false", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_dump_file": "", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fadvise": "true", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fail_eio": "true", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fd_cache_shards": "16", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fd_cache_size": "128", 2024-04-18T22:22:53.121 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fiemap": "false", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fiemap_threshold": "4096", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_inject_stall": "0", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_parallel": "false", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_trailing": "false", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_writeahead": "false", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_kill_at": "0", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-18T22:22:53.122 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs": "0", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size": "0", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_merge_threshold": "-10", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_odsync_write": "false", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_backend": "rocksdb", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_backend_path": "", 2024-04-18T22:22:53.123 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_thread_timeout": "60", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_threads": "2", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_punch_hole": "false", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-18T22:22:53.124 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_ops": "50", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_seek_data_hole": "false", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_sloppy_crc": "false", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_splice": "false", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_split_multiple": "2", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_split_rand_factor": "20", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_update_to": "1000", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-18T22:22:53.125 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_enable": "true", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_xfs_extsize": "false", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_zfs_snap": "false", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "fio_dir": "/tmp/fio", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a1b9e9bc-8f1f-4426-9796-fae392070eec", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_allow_other": "true", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_atomic_o_trunc": "true", 2024-04-18T22:22:53.126 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_big_writes": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_debug": "false", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_default_permissions": "false", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_disable_pagecache": "false", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_max_write": "0", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_multithreaded": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_require_active_mds": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_set_user_groups": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_move": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_read": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_write": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_use_invalidate_cb": "true", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-18T22:22:53.127 INFO:teuthology.orchestra.run.smithi032.stdout: "gss_target_name": "ceph", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_file": "", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_inject_failure": "0", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_interval": "5", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "host": "smithi032", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-18T22:22:53.128 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "inject_early_sigterm": "false", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "jaeger_agent_port": "6799", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "jaeger_tracing_enable": "false", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_aio": "true", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_align_min_size": "65536", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_block_align": "true", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_block_size": "4096", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_dio": "true", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_discard": "false", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_force_aio": "false", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_ignore_corruption": "false", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_max_write_bytes": "10485760", 2024-04-18T22:22:53.129 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_max_write_entries": "100", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_replay_from": "0", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_write_header_frequency": "0", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_zero_on_create": "false", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_prefetch_periods": "10", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_prezero_periods": "5", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_write_head_interval": "15", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "keyfile": "", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-18T22:22:53.130 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_backend": "rocksdb", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_default_stripe_size": "65536", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_fsck_on_mount": "false", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_max_bytes": "67108864", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_max_ops": "512", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_nid_prealloc": "1024", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_onode_map_size": "1024", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_sync_submit_transaction": "false", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_sync_transaction": "false", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "librados_thread_count": "2", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "lockdep": "false", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "lockdep_force_backtrace": "false", 2024-04-18T22:22:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: "log_coarse_timestamps": "true", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_flush_on_exit": "false", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_graylog_host": "127.0.0.1", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_graylog_port": "12201", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_max_new": "1000", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_max_recent": "10000", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_stderr_prefix": "", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_stop_at_utilization": "0.970000", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": "true", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_graylog": "false", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_journald": "false", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_stderr": "false", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_syslog": "false", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "max_rotating_auth_attempts": "10", 2024-04-18T22:22:53.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_action_on_write_error": "1", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_alternate_name_max": "8192", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_asio_thread_count": "2", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_export_pin": "true", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_dirs": "true", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_interval": "5", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_interval": "10", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_max": "-1", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_max_until": "-1", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_merge_size": "50", 2024-04-18T22:22:53.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_midchunk": "0.300000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_min_start": "0.200000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_minchunk": "0.001000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_mode": "0", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_need_max": "1.200000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_need_min": "0.800000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_overload_epochs": "2", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_bits": "3", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_size": "10000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-18T22:22:53.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_target_decay": "10.000000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_grace": "15.000000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_interval": "4.000000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_mid": "0.700000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_quiesce_delay": "0", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_release_free_interval": "10", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_reservation": "0.050000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-18T22:22:53.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_interval": "1", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_threshold": "262144", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_prealloc_inos": "1000", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_connect_bootstrapping": "false", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_damage_table_max_entries": "10000", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_auth_pins": "false", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_frag": "false", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_scatterstat": "false", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_subtrees": "false", 2024-04-18T22:22:53.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_decay_halflife": "5.000000", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_default_dir_hash": "2", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_defer_session_stale": "true", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_deny_all_reconnect": "false", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_keys_per_op": "16384", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_max_commit_size": "10", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_max_entries": "0", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_prefetch": "true", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_on_map": "false", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_early_reply": "true", 2024-04-18T22:22:53.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_enable_op_tracker": "true", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_enforce_unique_name": "true", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_random": "true", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_health_summarize_threshold": "10", 2024-04-18T22:22:53.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_health_dummy": "false", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_migrator_session_race": "false", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_join_fs": "", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_journal_format": "1", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_create_at": "0", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_export_at": "0", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_import_at": "0", 2024-04-18T22:22:53.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_expire_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_replay_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_link_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_mdstable_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_openc_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_rename_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_shutdown_at": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_event_large_threshold": "524288", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_events_per_segment": "1024", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_max_events": "-1", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_max_segments": "128", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_pause": "false", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_segment_size": "0", 2024-04-18T22:22:53.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_trim_threshold": "128", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_warn_factor": "2.000000", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_caps_per_client": "1048576", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_completed_flushes": "100000", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_completed_requests": "100000", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_export_size": "20971520", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_file_recover": "32", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_files": "64", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_ops": "8192", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-18T22:22:53.141 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_snaps_per_dir": "100", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_metrics_update_interval": "2", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_min_caps_per_client": "100", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_min_caps_working_set": "10000", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_numa_node": "-1", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_complaint_time": "30.000000", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_duration": "600", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_size": "20", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_slow_op_size": "20", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_log_threshold": "5", 2024-04-18T22:22:53.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_ping_grace": "15", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_ping_interval": "5", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_caps": "30000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_warning_threshold": "262144", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_replay_interval": "1.000000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_root_ino_gid": "0", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_root_ino_uid": "0", 2024-04-18T22:22:53.143 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_shutdown_check": "0", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_skip_ino": "0", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_max_uid": "4294967294", 2024-04-18T22:22:53.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_min_uid": "0", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_rstat": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_standby_replay_damaged": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_symlink_recovery": "true", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_thrash_exports": "0", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_thrash_fragments": "0", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_tick_interval": "5.000000", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_valgrind_exit": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_verify_backtrace": "1", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_verify_scatter": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_wipe_sessions": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool_debug": "false", 2024-04-18T22:22:53.145 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_device_bytes": "1073741824", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_page_set": "false", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_page_size": "65536", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_bytes": "134217728", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_messages": "512", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_disabled_modules": "", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_max_pg_num_change": "128", 2024-04-18T22:22:53.146 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mds_bytes": "134217728", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mds_messages": "128", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mon_bytes": "134217728", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mon_messages": "128", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_osd_bytes": "536870912", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_osd_messages": "8192", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_pool": "true", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_standby_modules": "true", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_stats_period": "5", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_stats_threshold": "5", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_tick_period": "2", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-18T22:22:53.147 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_allow_pool_delete": "true", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_allow_pool_size_one": "true", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_auth_validate_all_caps": "true", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_bytes": "104857600", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_directed_command_retry": "2", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_parallel": "3", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_log_interval": "1.000000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-18T22:22:53.148 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_ping_interval": "10.000000", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_target_rank": "-1", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_file_level": "debug", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_file": "true", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_journald": "false", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-18T22:22:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_bootstrap": "false", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_start": "false", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_trim": "true", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cpu_threads": "4", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_crush_min_required_version": "hammer", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_daemon_bytes": "419430400", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_avail_crit": "5", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_avail_warn": "5", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_size_warn": "16106127360", 2024-04-18T22:22:53.150 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_json": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_transactions": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_extra_checks": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_reef": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_squid": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_down_mkfs_grace": "120", 2024-04-18T22:22:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_down_uptime_grace": "60", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_election_default_strategy": "1", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_election_timeout": "5.000000", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ping_divisor": "2", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_enable_op_tracker": "true", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_fake_pool_delete": "false", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_force_quorum_join": "false", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_globalid_prealloc": "10000", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_detail_to_clog": "false", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_log_update_period": "5", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_max_detail": "50", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog": "true", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog_interval": "600", 2024-04-18T22:22:53.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_host": "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_host_override": "", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_initial_members": "", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease": "5.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_full_interval": "50", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_max": "10000", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_max_summary": "50", 2024-04-18T22:22:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_log_epochs": "500", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_osd": "10000", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_pg_per_osd": "10000", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_pool_pg_num": "65536", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_force_trim_to": "0", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_skip_sanity": "false", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_memory_autotune": "true", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_memory_target": "2147483648", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_beacon_grace": "30", 2024-04-18T22:22:53.154 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_digest_period": "5", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_inactive_grace": "60", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_min_osdmap_epochs": "500", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_complaint_time": "30", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_duration": "600", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_size": "20", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_slow_op_size": "20", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_log_threshold": "5", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-18T22:22:53.155 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_in": "false", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_cache_size": "500", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_down_out_interval": "600", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_force_trim_to": "0", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-18T22:22:53.156 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_down_reporters": "2", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_report_timeout": "900", 2024-04-18T22:22:53.157 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_stuck_threshold": "60", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-18T22:22:53.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_probe_timeout": "2.000000", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_max_change": "0.050000", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_max_osds": "4", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_interval": "86400", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_max_keys": "100", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_timeout": "300", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_session_timeout": "300", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_smart_report_timeout": "5", 2024-04-18T22:22:53.159 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stat_smooth_intervals": "6", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_pool_min_size": "2", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_pool_size": "4", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_debug": "false", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_provider_kill_at": "0", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_requester_kill_at": "0", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_timeout": "60.000000", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_target_pg_per_osd": "100", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_tick_interval": "5", 2024-04-18T22:22:53.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_timecheck_interval": "300.000000", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_use_min_delay_socket": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_older_version_delay": "604800", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_misplaced": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_older_version": "true", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-18T22:22:53.161 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "monmap": "", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-18T22:22:53.162 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-18T22:22:53.166 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-18T22:22:53.166 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-18T22:22:53.166 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-18T22:22:53.166 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_tracing_enabled": "false", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_op_threads": "3", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_cm": "false", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_device_name": "", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_dscp": "96", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_local_gid": "", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_port_num": "1", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-18T22:22:53.167 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_sl": "3", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_support_srq": "true", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_type": "ib", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_reap_threshold": "5", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_before_connect": "false", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_ipv4": "true", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_ipv6": "false", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_msgr1": "true", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_msgr2": "true", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_port_max": "7568", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_port_min": "6800", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_retry_count": "3", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_retry_delay": "5", 2024-04-18T22:22:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_client": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mds": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mgr": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mon": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_osd": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_client_mode": "crc secure", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_cluster_mode": "crc secure", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_cluster_type": "", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_compress_secure": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_connection_idle_timeout": "900", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_connection_ready_timeout": "10", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_crc_data": "true", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_crc_header": "true", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_bad_msg": "false", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_bug": "true", 2024-04-18T22:22:53.169 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_old_message": "true", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_skipped_message": "false", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_coremask": "0xF", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_enable_tso": "true", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hugepages": "", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-18T22:22:53.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_lro": "true", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_memory_channel": "4", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_pmd": "", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_port_id": "0", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dump_on_send": "false", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_initial_backoff": "0.200000", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_max": "1.000000", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_type": "", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_network_congestion": "0", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_socket_failures": "0", 2024-04-18T22:22:53.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_learn_addr_from_peer": "true", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_max_accept_failures": "4", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_max_backoff": "15.000000", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_client_mode": "secure crc", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_service_mode": "secure crc", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compress_min_size": "1024", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compress_mode": "none", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_pq_min_cost": "65536", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_public_type": "", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_service_mode": "crc secure", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_listen_backlog": "512", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_nodelay": "true", 2024-04-18T22:22:53.172 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_rcvbuf": "0", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_type": "async+posix", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "no_config_file": "false", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_completion_locks_per_session": "32", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inflight_ops": "1024", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_tick_interval": "5.000000", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_timeout": "10.000000", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objectstore_blackhole": "false", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-18T22:22:53.173 INFO:teuthology.orchestra.run.smithi032.stdout: "openssl_engine_opts": "", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_delay_time": "5.000000", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_hist_halflife": "1000", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_max_low_ops": "2", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_max_ops": "4", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_slop": "0.020000", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_asio_thread_count": "2", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_async_recovery_min_cost": "100", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_scan_max": "512", 2024-04-18T22:22:53.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_scan_min": "64", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_degraded": "false", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_peering": "false", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_unfound": "true", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_beacon_report_interval": "300", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_duration": "30", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_max_block_size": "67108864", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_blkin_trace_all": "false", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_check_for_log_corruption": "false", 2024-04-18T22:22:53.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_update_on_start": "true", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_message_cap": "256", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_message_size_cap": "524288000", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_op_priority": "63", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_watch_timeout": "30", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_command_max_records": "256", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_compact_on_start": "false", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_update_on_start": "true", 2024-04-18T22:22:53.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_update_weight_set": "true", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_feed_pullee": "-1", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_misdirected_ops": "false", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_no_acting_change": "false", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_no_purge_strays": "false", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_op_order": "false", 2024-04-18T22:22:53.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_shutdown": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_trim_objects": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_snaps": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_keys": "1024", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-18T22:22:53.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_stride": "524288", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_default_notify_timeout": "30", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep": "0.000000", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_discard_disconnected_ops": "true", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_enable_op_tracker": "true", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-18T22:22:53.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_info": "true", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown": "true", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_function_tracing": "false", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_grace": "20", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_interval": "6", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_peers": "10", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_size": "2000", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_stale": "600", 2024-04-18T22:22:53.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_max_size": "100000", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_min_size": "1000", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal_size": "5120", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_kill_backfill_at": "0", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_cache_size": "50", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_dedup": "true", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_message_max": "40", 2024-04-18T22:22:53.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_share_max_epochs": "40", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_attr_name_len": "100", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_attr_size": "0", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_backfills": "1", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_markdown_count": "5", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_markdown_period": "600", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_name_len": "2048", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_namespace_len": "256", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_size": "134217728", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_blocked_by": "16", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_log_entries": "10000", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-18T22:22:53.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pgls": "1024", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_push_cost": "8388608", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_push_objects": "10", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_scrubs": "3", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_trimming_pgs": "2", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_write_op_reply_len": "64", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_write_size": "90", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-18T22:22:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_profile": "balanced", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_base": "805306368", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_cache_min": "134217728", 2024-04-18T22:22:53.184 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target": "4294967296", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target_autotune": "false", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_min_pg_log_entries": "250", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_min_recovery_priority": "0", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_report_interval": "5", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-18T22:22:53.185 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_num_cache_shards": "32", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_num_op_tracker_shard": "32", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_auto_affinity": "true", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_node": "-1", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_prefer_iface": "true", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objecter_finishers": "1", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore": "bluestore", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore_fuse": "false", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore_tracing": "false", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_complaint_time": "30.000000", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_duration": "600", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_size": "20", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_slow_op_size": "20", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-18T22:22:53.186 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_log_threshold": "5", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards": "0", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards_hdd": "5", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards_ssd": "8", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_pq_min_cost": "65536", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_queue_cut_off": "high", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_thread_timeout": "15", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_open_classes_on_start": "true", 2024-04-18T22:22:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_os_flags": "0", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_peering_op_priority": "255", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_delete_cost": "1048576", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_delete_priority": "5", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_trim_max": "10000", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_trim_min": "100", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-18T22:22:53.188 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_crimson": "false", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flags": "0", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-18T22:22:53.189 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_min_size": "0", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pg_num": "32", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pgp_num": "0", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_read_ratio": "70", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_size": "2", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_type": "replicated", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_push_per_object_cost": "1000", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recover_clone_overlap": "true", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-18T22:22:53.190 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_cost": "20971520", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active": "0", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_single_start": "1", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_op_priority": "3", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_priority": "5", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep": "0.000000", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-18T22:22:53.191 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_repair_during_recovery": "false", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_requested_scrub_priority": "120", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_auto_repair": "false", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_begin_hour": "0", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_begin_week_day": "0", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_chunk_max": "25", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_chunk_min": "5", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_cost": "52428800", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_during_recovery": "false", 2024-04-18T22:22:53.192 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_end_hour": "0", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_end_week_day": "0", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_event_cost": "4096", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_invalid_stats": "true", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_max_preemptions": "5", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_priority": "5", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_reservation_timeout": "300000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_sleep": "0.000000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_slow_reservation_response": "30000", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-18T22:22:53.193 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_skip_data_digest": "false", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_smart_report_timeout": "5", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_cost": "1048576", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_priority": "5", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_target_transaction_size": "30", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-18T22:22:53.194 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tracing": "false", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_use_stale_snap": "false", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "osdc_blkin_trace_all": "false", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_kill_at": "0", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_max_join_drift": "10", 2024-04-18T22:22:53.195 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_min": "500", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_min_wait": "0.050000", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_propose_interval": "1.000000", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_max": "500", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_min": "250", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_stash_full_interval": "25", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_trim_max": "500", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_trim_min": "250", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "perf": "true", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin_dir": "/usr/lib/ceph", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "-", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrv": "", 2024-04-18T22:22:53.196 INFO:teuthology.orchestra.run.smithi032.stdout: "public_bind_addr": "-", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "public_network": "", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "public_network_interface": "", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "qat_compressor_enabled": "false", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "qat_compressor_session_max_number": "256", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_mon_op_timeout": "0", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_osd_op_timeout": "0", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_tracing": "false", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_atime_update_interval": "60", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_balance_parent_reads": "false", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_balance_snap_reads": "false", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blkin_trace_all": "false", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-18T22:22:53.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache": "true", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_policy": "writearound", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_size": "33554432", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_clone_copy_on_read": "false", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_compression_hint": "none", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_concurrent_management_ops": "10", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_clone_format": "auto", 2024-04-18T22:22:53.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_data_pool": "", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_features": "61", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_format": "2", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_map_options": "", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_order": "22", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_pool": "rbd", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_stripe_count": "0", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_stripe_unit": "0", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_enable_alloc_hint": "true", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_io_scheduler": "simple", 2024-04-18T22:22:53.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_order": "24", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_pool": "", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_splay_width": "4", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_localize_parent_reads": "false", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_localize_snap_reads": "false", 2024-04-18T22:22:53.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-18T22:22:53.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-18T22:22:53.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mtime_update_interval": "60", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_non_blocking_aio": "true", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_op_thread_timeout": "60", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_op_threads": "1", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_parent_cache_enabled": "false", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_plugins": "", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_burst": "0", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_limit": "0", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_exclude_ops": "0", 2024-04-18T22:22:53.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_burst": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_limit": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-18T22:22:53.204 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_read_from_replica_policy": "default", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_skip_partial_discard": "true", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_tracing": "false", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_validate_names": "true", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_validate_pool": "true", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "restapi_base_url": "", 2024-04-18T22:22:53.205 INFO:teuthology.orchestra.run.smithi032.stdout: "restapi_log_level": "", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_acl_grants_max_num": "100", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_admin_entry": "admin", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_backend_store": "rados", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_barbican_url": "", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_beast_enable_async": "true", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_counters_cache": "false", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-18T22:22:53.206 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_enabled": "true", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_expiry_interval": "900", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_lru_size": "10000", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_config_store": "rados", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_content_length_compat": "false", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_copy_obj_progress": "true", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cors_rules_max_num": "100", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cross_domain_policy": "", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_addr": "", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-18T22:22:53.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_password": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_username": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_require_ssl": "true", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-18T22:22:53.208 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_addr": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_auth": "token", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_namespace": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_prefix": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-18T22:22:53.209 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_token_file": "", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_buffersize": "524288", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_low_speed_time": "300", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-18T22:22:53.210 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d4n_port": "6379", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_changes_size": "1000", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_num_shards": "128", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_window": "30", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-18T22:22:53.211 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-18T22:22:53.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dns_name": "", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dns_s3website_name": "", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dynamic_resharding": "true", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_gc_threads": "true", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_lc_threads": "true", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_ops_log": "false", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_quota_threads": "true", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_static_website": "false", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_usage_log": "false", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enforce_swift_acls": "true", 2024-04-18T22:22:53.213 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_exit_timeout_secs": "120", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_expose_bucket": "false", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_extended_http_attrs": "", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_filter": "none", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_frontends": "beast port=7480", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_deferred": "50", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_objs": "32", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_processor_period": "3600", 2024-04-18T22:22:53.214 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_init_timeout": "300", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_kafka_connection_idle": "300", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_domain": "", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_password": "", 2024-04-18T22:22:53.215 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_password_path": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_project": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_tenant": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_token": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_token_path": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_user": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_api_version": "2", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_domain": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_password": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_project": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_user": "", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-18T22:22:53.216 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_url": "", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_debug_interval": "-1", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_lock_max_time": "90", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_objs": "32", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_rules": "1000", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_worker": "3", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_thread_delay": "0", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_dnattr": "uid", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_searchfilter": "", 2024-04-18T22:22:53.217 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_uri": "", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_http_headers": "", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_object_name_utc": "false", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attr_name_len": "0", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attr_size": "0", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-18T22:22:53.218 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_chunk_size": "4194304", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_listing_results": "1000", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_notify_retries": "10", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_put_param_size": "1048576", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_put_size": "5368709120", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_slo_entries": "1000", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_md_log_max_shards": "64", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-18T22:22:53.219 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_mp_lock_max_time": "600", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_max_gc": "300", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-18T22:22:53.220 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_num_async_rados_threads": "32", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_num_control_oids": "8", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_numa_node": "-1", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_chunk_size": "100", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_gc_interval": "600", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_thread_timeout": "600", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_tracing": "false", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_token": "", 2024-04-18T22:22:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_url": "", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_verify_ssl": "true", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_rados": "false", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_socket_path": "", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_push_interval": "2.000000", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_posix_cache_lanes": "3", 2024-04-18T22:22:53.222 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_posix_cache_partitions": "3", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_print_continue": "true", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_tracing": "false", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm": "", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm_id": "", 2024-04-18T22:22:53.223 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_region": "", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_request_uri": "", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_batch_size": "64", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_max_aio": "128", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_num_logs": "16", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_thread_interval": "600", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_resolve_cname": "false", 2024-04-18T22:22:53.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_run_sync_thread": "true", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_scheduler_type": "throttler", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_script_uri": "", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_service_provider_name": "", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-18T22:22:53.225 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_client_id": "", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_client_secret": "", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_entry": "sts", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_key": "sts", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_min_session_duration": "900", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_token_introspection_url": "", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_account_in_url": "false", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_auth_entry": "auth", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_auth_url": "", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_custom_header": "", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_need_stats": "true", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_tenant_name": "", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_token_expiration": "86400", 2024-04-18T22:22:53.226 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_url": "", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_url_prefix": "swift", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_lease_period": "120", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-18T22:22:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_thread_pool_size": "512", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_comment": "", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_createby": "", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_encoding": "", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_flag": "false", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_origin": "", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_tracker": "", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_trust_forwarded_https": "false", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-18T22:22:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_max_shards": "32", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_max_user_shards": "1", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_use_opa_authz": "false", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_counters_cache": "false", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_max_buckets": "1000", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_policies_max_num": "100", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_unique_email": "true", 2024-04-18T22:22:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_verify_ssl": "true", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone": "", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone_id": "", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup": "", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup_id": "", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgwlc_auto_session_clear": "true", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_block_size": "4096", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-18T22:22:53.230 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_size": "536870912", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_index_type": "binary_search", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_partition_filters": "false", 2024-04-18T22:22:53.231 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_perf": "false", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "run_dir": "/var/run/ceph", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_block_create": "true", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_cache_lru_size": "67108864", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_default_max_object_size": "16777216", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_device_size": "53687091200", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_capacity": "16", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-18T22:22:53.232 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_main_device_type": "SSD", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_segment_size": "67108864", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "set_keepcaps": "false", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "setgroup": "", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "setuser": "", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "setuser_match_path": "", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "thp": "false", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "threadpool_default_timeout": "60", 2024-04-18T22:22:53.233 INFO:teuthology.orchestra.run.smithi032.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-18T22:22:53.234 INFO:teuthology.orchestra.run.smithi032.stdout: "throttler_perf_counter": "true" 2024-04-18T22:22:53.234 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:22:53.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-18T22:22:53.380 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a: {} 2024-04-18T22:22:53.380 INFO:teuthology.orchestra.run.smithi032.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T22:22:53.389 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b: {} 2024-04-18T22:22:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T22:22:53.398 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c: {} 2024-04-18T22:22:53.399 INFO:teuthology.orchestra.run.smithi032.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T22:22:53.415 INFO:tasks.thrashosds.thrasher:Unable to test ceph-objectstore-tool, powercycle testing 2024-04-18T22:22:53.415 INFO:teuthology.run_tasks:Running task radosbench... 2024-04-18T22:22:53.440 INFO:tasks.radosbench:Beginning radosbench... 2024-04-18T22:22:53.440 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-18T22:22:53.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-04-18T22:22:53.441 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-18T22:22:53.441 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:22:53.441 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:22:53.441 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-04-18T22:22:53.442 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-18T22:22:53.442 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-18T22:22:53.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-04-18T22:22:53.442 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-18T22:22:53.443 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-18T22:22:53.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:22:53.443 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-18T22:22:53.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:22:53.444 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-18T22:22:53.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:22:53.446 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:53.444+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:53.545 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:53.544+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:53.574 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:22:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-18T22:22:53.574 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:22:53.576 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:22:53.576 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:22:53.577 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-04-18T22:22:53.577 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:22:53.579 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:22:53.580 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:22:53.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:22:53.600 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:22:53.600 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:22:53.620 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:22:53.620 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:22:53.647 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:53.648+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:53.710 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:53.710 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:53.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:22:53.779 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:53.775+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:53.844 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:53.844 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:53.848 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:53.846+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:53.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:22:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'unique_pool_0' created 2024-04-18T22:22:53.939 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:22:53.948 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:53.947+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:53.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-18T22:22:53.989 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:53.989 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:22:54.050 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:54.051+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:54.124 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:54.124 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.139 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:22:54.192 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:54.186+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:54.251 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:54.250+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.276 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:22:54.352 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:54.350+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:54.407 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:54.407 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:22:54.453 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:54.455+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:54.538 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:54.538 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:22:54.554 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:54.554+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:54.655 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:54.655+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:22:54.756 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:54.755+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:54.818 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:54.819 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:54.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:22:54.877 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:54.879+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-18T22:22:54.961 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:54.956+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:55.008 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:55.009 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:55.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:22:55.069 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:55.071+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:55.144 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:55.144 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:55.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:22:55.161 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:55.158+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:55.261 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:55.260+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:55.289 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:55.289 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:55.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:22:55.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:55.360+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:55.426 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:55.426 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:55.439 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:22:55.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:55.460+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:55.558 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:55.558 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:55.563 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:55.558+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:55.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:22:55.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:22:55.664 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:55.663+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:55.703 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:55.703 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:55.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:22:55.765 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:55.760+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:55.799 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:22:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:22:55.820 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:22:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:22:55.841 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:22:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:22:55.842 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:22:55.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:22:55.842 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:22:55.842 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:22:55.842 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:22:55.855 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:22:55.866 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:55.862+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:55.967 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:55.964+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:55.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:22:55.976 DEBUG:teuthology.orchestra.run.smithi032:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 60 write' 2024-04-18T22:22:56.017 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:22:56.017 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:22:56.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:22:56.017 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4144.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e10)", 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 33.555362877999997, 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025591307000000001, 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4144", 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1049496302", 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 141 2024-04-18T22:22:56.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461622+0000", 2024-04-18T22:22:56.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9400000000000004e-07 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461627+0000", 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6559999999999997e-06 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461638+0000", 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0594000000000001e-05 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461645+0000", 2024-04-18T22:22:56.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2099999999999996e-06 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480273+0000", 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018627812000000001 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480507+0000", 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00023432800000000001 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480828+0000", 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032085799999999998 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.482445+0000", 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001617081 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487064+0000", 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046189179999999996 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487149+0000", 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4575999999999998e-05 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487212+0000", 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3479999999999995e-05 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:6 3.10 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.902027056000001, 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027601216000000001, 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:22:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:22:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114960+0000", 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.88e-06 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114981+0000", 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0877999999999999e-05 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115008+0000", 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7617999999999999e-05 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115031+0000", 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2302000000000001e-05 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121370+0000", 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063390540000000002 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121632+0000", 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000262539 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121921+0000", 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028841099999999997 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135317+0000", 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013396105 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142295+0000", 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0069786539999999999 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142445+0000", 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000149968 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142558+0000", 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011280700000000001 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.030 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901914495, 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:22:56.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:22:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901845897000001, 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.037 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:22:56.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:22:56.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901750051, 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.043 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:22:56.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:22:56.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901693169000001, 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:22:56.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:22:56.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.055 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901633496999999, 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:22:56.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:22:56.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901575534999999, 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.061 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:22:56.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:22:56.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:22:56.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:22:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:22:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:22:56.069 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:15 3.14 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901510917, 2024-04-18T22:22:56.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027152815, 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115474+0000", 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8700000000000004e-07 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115482+0000", 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2560000000000002e-06 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115500+0000", 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8224999999999999e-05 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115514+0000", 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3648999999999999e-05 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115656+0000", 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000141865 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115959+0000", 2024-04-18T22:22:56.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030357600000000001 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116299+0000", 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033980600000000001 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126282+0000", 2024-04-18T22:22:56.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0099824879999999994 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142386+0000", 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016104535999999999 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142517+0000", 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013121000000000001 2024-04-18T22:22:56.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142626+0000", 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108317 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:16 3.1c 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:22:56.078 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901419381, 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027983006000000001, 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:22:56.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115566+0000", 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.127e-06 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115573+0000", 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0559999999999999e-06 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115591+0000", 2024-04-18T22:22:56.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8615e-05 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115608+0000", 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6815000000000001e-05 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117673+0000", 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020651689999999999 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.118082+0000", 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040828099999999998 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.118459+0000", 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037744699999999999 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126370+0000", 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079108990000000007 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143152+0000", 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016782488000000002 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143434+0000", 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000281988 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143548+0000", 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113121 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.083 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901355843000001, 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:22:56.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:22:56.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:22:56.090 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901298914000002, 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:22:56.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:22:56.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901203028000001, 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:22:56.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:22:56.102 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901140699999999, 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:22:56.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:22:56.108 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.901068221999999, 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:22:56.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:22:56.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:22:56.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:2 3.f 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.90100799, 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026491475, 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.06e-06 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115983+0000", 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2820000000000002e-06 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116005+0000", 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1525000000000001e-05 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116018+0000", 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3372e-05 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116290+0000", 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000271453 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116676+0000", 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038682899999999999 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117663+0000", 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00098642700000000001 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125645+0000", 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079826340000000006 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142118+0000", 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016472647 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142261+0000", 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014285799999999999 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142467+0000", 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000206388 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.118 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.900915754, 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:22:56.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:22:56.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:22:56.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.125 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.900837031000002, 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:22:56.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:22:56.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:22:56.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:22:56.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.899316174999999, 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.132 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:22:56.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:22:56.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.137 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.899217823000001, 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:56.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:56.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:56.142 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:22:56.144 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:56.140+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:56.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:22:56.145 INFO:teuthology.run_tasks:Running task admin_socket... 2024-04-18T22:22:56.156 DEBUG:tasks.admin_socket:Running admin socket tests on client.0 2024-04-18T22:22:56.156 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -- /home/ubuntu/cephtest/admin_socket_client.0 && while test '!' -e /var/run/ceph/ceph-client.0.asok ; do sleep 1 ; done 2024-04-18T22:22:56.188 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:hints = 1 2024-04-18T22:22:56.188 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 60 seconds or 0 objects 2024-04-18T22:22:56.188 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Object prefix: benchmark_data_smithi032_22800 2024-04-18T22:22:56.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-18T22:22:56.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 0 0 0 0 0 0 - 0 2024-04-18T22:22:56.245 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:56.243+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:310 5.2 e21/19 5:42486003:::benchmark_data_smithi032_22800_object309:head v 21'17, mlcod=21'15)", 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.326232+0000", 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000269521, 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030067800000000002, 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:22:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326232+0000", 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326232+0000", 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326233+0000", 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7500000000000001e-07 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326370+0000", 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000136988 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326377+0000", 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9039999999999992e-06 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326385+0000", 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2579999999999998e-06 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326435+0000", 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0630000000000001e-05 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326460+0000", 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4358e-05 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:311 5.b e21/19 5:de494f56:::benchmark_data_smithi032_22800_object310:head v 21'21, mlcod=21'18)", 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.326451+0000", 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.0526999999999999e-05, 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018244100000000001, 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326451+0000", 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326451+0000", 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326452+0000", 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5899999999999996e-07 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326515+0000", 2024-04-18T22:22:56.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3320999999999999e-05 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326523+0000", 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8280000000000003e-06 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326532+0000", 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8319999999999995e-06 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326574+0000", 2024-04-18T22:22:56.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2463999999999999e-05 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326588+0000", 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3291e-05 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:301 5.a 5:5ce41e48:::benchmark_data_smithi032_22800_object300:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.320827+0000", 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0056748129999999999, 2024-04-18T22:22:56.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0059110380000000004, 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 301 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320827+0000", 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320827+0000", 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320828+0000", 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.108e-06 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320874+0000", 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6297000000000001e-05 2024-04-18T22:22:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320887+0000", 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3168e-05 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320902+0000", 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4139e-05 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.320966+0000", 2024-04-18T22:22:56.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4510000000000004e-05 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321084+0000", 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117586 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321188+0000", 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103878 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.323197+0000", 2024-04-18T22:22:56.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020097460000000002 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:303 5.a 5:535725e5:::benchmark_data_smithi032_22800_object302:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.321497+0000", 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0050044119999999997, 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005366984, 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 303 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321497+0000", 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321497+0000", 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321498+0000", 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5099999999999998e-07 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321548+0000", 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9614999999999998e-05 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321560+0000", 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1844999999999999e-05 2024-04-18T22:22:56.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321567+0000", 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5190000000000003e-06 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321615+0000", 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7988000000000002e-05 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321776+0000", 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016066400000000001 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321923+0000", 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014729200000000001 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.323201+0000", 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001278335 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:304 5.a 5:5b2c84bd:::benchmark_data_smithi032_22800_object303:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.321583+0000", 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0049181939999999999, 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054157099999999998, 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 304 2024-04-18T22:22:56.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321583+0000", 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321583+0000", 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321584+0000", 2024-04-18T22:22:56.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5899999999999996e-07 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321644+0000", 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9689999999999999e-05 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321653+0000", 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.397e-06 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.321660+0000", 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5379999999999999e-06 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.322503+0000", 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000843007 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.322642+0000", 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013967 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.322763+0000", 2024-04-18T22:22:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012057600000000001 2024-04-18T22:22:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.323598+0000", 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00083509700000000003 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:308 5.5 5:aead053b:::benchmark_data_smithi032_22800_object307:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.324942+0000", 2024-04-18T22:22:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0015600729999999999, 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021957050000000001, 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 308 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.324942+0000", 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.324942+0000", 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.324943+0000", 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1689999999999999e-06 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325045+0000", 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102019 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325058+0000", 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2918000000000001e-05 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325072+0000", 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4467000000000001e-05 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325129+0000", 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6827000000000003e-05 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325289+0000", 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015971099999999999 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325561+0000", 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027217300000000001 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326685+0000", 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001124014 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:312 5.6 5:636dbe38:::benchmark_data_smithi032_22800_object311:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.325915+0000", 2024-04-18T22:22:56.352 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00058678699999999996, 2024-04-18T22:22:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012876770000000001, 2024-04-18T22:22:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.353 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 312 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325915+0000", 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325915+0000", 2024-04-18T22:22:56.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325916+0000", 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9100000000000003e-07 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325961+0000", 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5198000000000003e-05 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325978+0000", 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7518000000000001e-05 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.325989+0000", 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0380999999999999e-05 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326072+0000", 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3086999999999999e-05 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326224+0000", 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015261699999999999 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326338+0000", 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011353699999999999 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.327148+0000", 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080953100000000005 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:309 5.0 e21/19 5:0cb8af03:::benchmark_data_smithi032_22800_object308:head v 21'31, mlcod=21'27)", 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.326188+0000", 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000313233, 2024-04-18T22:22:56.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010779019999999999, 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326188+0000", 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326188+0000", 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326189+0000", 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7499999999999998e-07 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326309+0000", 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120041 2024-04-18T22:22:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326319+0000", 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.713e-06 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326326+0000", 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5819999999999998e-06 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326388+0000", 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1803000000000004e-05 2024-04-18T22:22:56.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.326414+0000", 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6313999999999999e-05 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.362 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:22:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 8 2024-04-18T22:22:56.363 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:22:56.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:22:56.365 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:56.362+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:56.465 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:56.462+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:56.565 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:56.560+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:56.573 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:22:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:22:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:22:56.573 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:22:56.573 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:22:56.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:22:56.666 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:56.667+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:56.751 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:22:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.636317795, 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.636185303000001, 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.636113717000001, 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:22:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:22:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:22:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.635985464000001, 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:22:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:22:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.635876137, 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:22:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:21 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.635706022000001, 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.027074913999999999, 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116210+0000", 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9190000000000002e-06 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116218+0000", 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.994e-06 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116239+0000", 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0491000000000001e-05 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116260+0000", 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1118e-05 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119672+0000", 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034115410000000001 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119685+0000", 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3866000000000001e-05 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.132805+0000", 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013119445 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133030+0000", 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022555700000000001 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133216+0000", 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018553999999999999 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140260+0000", 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070438760000000001 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.142993+0000", 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027334389999999998 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.143165+0000", 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000172165 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.143283+0000", 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117963 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.635619800000001, 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:22:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.635532313999999, 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:22:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:22:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.635454328000002, 2024-04-18T22:22:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:22:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(mds.0.3:37 3.13 e17/15 3:cd5a64a3:::100.00000000:head v 17'1, mlcod=0'0)", 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 24.625695744000001, 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014004429000000001, 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126220+0000", 2024-04-18T22:22:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1299999999999998e-07 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126247+0000", 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7328e-05 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126267+0000", 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9655000000000001e-05 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126284+0000", 2024-04-18T22:22:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7075000000000002e-05 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126429+0000", 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014528699999999999 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126491+0000", 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1587999999999994e-05 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140139+0000", 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013648476 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140223+0000", 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4107e-05 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:3 5.6 5:62098041:::benchmark_data_smithi032_22800_object2:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:22:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55906022799999999, 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012453989, 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:22:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192857+0000", 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5780000000000001e-06 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193049+0000", 2024-04-18T22:22:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019216900000000001 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193084+0000", 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5265999999999997e-05 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193098+0000", 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3295000000000001e-05 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193303+0000", 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020540599999999999 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193313+0000", 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0093e-05 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200364+0000", 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070508469999999998 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200467+0000", 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010301099999999999 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200628+0000", 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016122700000000001 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202430+0000", 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001801551 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205199+0000", 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027694490000000002 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205265+0000", 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5505000000000004e-05 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205308+0000", 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3591999999999999e-05 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:7 5.6 5:6a6275e8:::benchmark_data_smithi032_22800_object6:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55869045900000003, 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013282308, 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193225+0000", 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.009e-06 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193457+0000", 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000232308 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193472+0000", 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5101e-05 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193484+0000", 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1990999999999999e-05 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193553+0000", 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8149000000000001e-05 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193559+0000", 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5339999999999996e-06 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201146+0000", 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075871929999999999 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201251+0000", 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104695 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201374+0000", 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012274200000000001 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202530+0000", 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011561010000000001 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206367+0000", 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038374400000000001 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206448+0000", 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0486000000000002e-05 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206506+0000", 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8559e-05 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:8 5.5 5:a946a8f1:::benchmark_data_smithi032_22800_object7:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193502+0000", 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55841245399999995, 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011748221999999999, 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 8 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193502+0000", 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193502+0000", 2024-04-18T22:22:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193503+0000", 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0800000000000003e-07 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193596+0000", 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2651000000000005e-05 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193606+0000", 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0583e-05 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193612+0000", 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7540000000000003e-06 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193725+0000", 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112884 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193733+0000", 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2719999999999997e-06 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.199417+0000", 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0056835419999999998 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.199626+0000", 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020924100000000001 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.199753+0000", 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012742799999999999 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201341+0000", 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015872810000000001 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205132+0000", 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037917200000000002 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205192+0000", 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9700000000000001e-05 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205250+0000", 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8257999999999997e-05 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:10 5.5 5:ac01b574:::benchmark_data_smithi032_22800_object9:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55828280399999997, 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011974561999999999, 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 10 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.37e-07 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193690+0000", 2024-04-18T22:22:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7331999999999999e-05 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193697+0000", 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2139999999999999e-06 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193703+0000", 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.891e-06 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193760+0000", 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7358999999999997e-05 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193765+0000", 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8670000000000002e-06 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200394+0000", 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066286399999999999 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200540+0000", 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000146427 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200705+0000", 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000164871 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202046+0000", 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013406469999999999 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205489+0000", 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034432730000000002 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205555+0000", 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6476000000000005e-05 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205606+0000", 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0927999999999998e-05 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14 5.6 5:6abb6cd9:::benchmark_data_smithi032_22800_object13:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55820022400000002, 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012915045999999999, 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193715+0000", 2024-04-18T22:22:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6699999999999999e-07 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193774+0000", 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9286000000000001e-05 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193783+0000", 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6000000000000007e-06 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193788+0000", 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3639999999999998e-06 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193837+0000", 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8656000000000003e-05 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193841+0000", 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8489999999999999e-06 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201856+0000", 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0080156819999999997 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201955+0000", 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8970999999999998e-05 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202087+0000", 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013226199999999999 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202906+0000", 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00081854200000000001 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206559+0000", 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036528469999999999 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206600+0000", 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0794999999999998e-05 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206629+0000", 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9725000000000001e-05 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:15 5.c 5:330784a8:::benchmark_data_smithi032_22800_object14:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193799+0000", 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55811572499999995, 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010752228000000001, 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 15 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193799+0000", 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193799+0000", 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193799+0000", 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4199999999999995e-07 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193869+0000", 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9753999999999997e-05 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193880+0000", 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1080999999999999e-05 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193886+0000", 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5339999999999999e-06 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193929+0000", 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3219000000000001e-05 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193933+0000", 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6729999999999998e-06 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.198924+0000", 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0049914649999999996 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.199084+0000", 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015992799999999999 2024-04-18T22:22:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.199262+0000", 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017778299999999999 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201748+0000", 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002486565 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.204476+0000", 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027274819999999998 2024-04-18T22:22:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.204517+0000", 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1183999999999997e-05 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.204551+0000", 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3918000000000003e-05 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:16 5.3 5:c7c176a5:::benchmark_data_smithi032_22800_object15:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193899+0000", 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.55801590999999995, 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0094292990000000004, 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 16 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193899+0000", 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193899+0000", 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193899+0000", 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.68e-07 2024-04-18T22:22:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193971+0000", 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1860999999999995e-05 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193982+0000", 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1044e-05 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193989+0000", 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.058e-06 2024-04-18T22:22:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.194055+0000", 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6125999999999997e-05 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.194061+0000", 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3349999999999997e-06 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.198635+0000", 2024-04-18T22:22:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045744929999999998 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.198822+0000", 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018708200000000001 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.199020+0000", 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019745599999999999 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200214+0000", 2024-04-18T22:22:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011946229999999999 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.203250+0000", 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030357869999999999 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.203293+0000", 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2976999999999999e-05 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.203328+0000", 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4789000000000002e-05 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:127 5.5 5:aad99fce:::benchmark_data_smithi032_22800_object126:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.246631+0000", 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.50528332200000003, 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010243118000000001, 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 127 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246631+0000", 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246631+0000", 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246632+0000", 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0330000000000001e-06 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246713+0000", 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0311999999999999e-05 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246726+0000", 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3875e-05 2024-04-18T22:22:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246737+0000", 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0361e-05 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.246839+0000", 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010204600000000001 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247165+0000", 2024-04-18T22:22:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032654 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247469+0000", 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030395200000000002 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.250189+0000", 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027192980000000002 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.256721+0000", 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006532131 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.256805+0000", 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4592000000000006e-05 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.256874+0000", 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8978000000000003e-05 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.856 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:124 5.e e21/19 5:73ef8cd7:::benchmark_data_smithi032_22800_object123:head v 21'12, mlcod=21'10)", 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.247175+0000", 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.50473985300000002, 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0095925539999999997, 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247175+0000", 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247175+0000", 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247176+0000", 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0100000000000001e-06 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247231+0000", 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5421999999999999e-05 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247254+0000", 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2563999999999999e-05 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247270+0000", 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6566999999999999e-05 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247328+0000", 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7608999999999997e-05 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247342+0000", 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4280999999999999e-05 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.256689+0000", 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093467240000000007 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.256767+0000", 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8376999999999999e-05 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:125 5.b e21/19 5:d87693cd:::benchmark_data_smithi032_22800_object124:head v 21'8, mlcod=21'3)", 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.247530+0000", 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.504384622, 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0095536839999999998, 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:56.861 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247530+0000", 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247530+0000", 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247531+0000", 2024-04-18T22:22:56.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4199999999999995e-07 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247778+0000", 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00024714000000000002 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247790+0000", 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1895e-05 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247798+0000", 2024-04-18T22:22:56.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5699999999999993e-06 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247877+0000", 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8687000000000001e-05 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.247903+0000", 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6446999999999999e-05 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.257025+0000", 2024-04-18T22:22:56.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091217699999999995 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.257084+0000", 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8433000000000001e-05 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:22:56.865 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:22:56.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:22:56.867 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:56.867+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:22:56.968 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:56.966+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:57.032 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:22:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:22:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4586.0:3019 5.a e22/19 5:5556601f:::benchmark_data_smithi032_22800_object3018:head v 22'194, mlcod=22'190)", 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.032176+0000", 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001399069, 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001422043, 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032176+0000", 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032176+0000", 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032177+0000", 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7599999999999997e-07 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032232+0000", 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5355000000000001e-05 2024-04-18T22:22:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032241+0000", 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6070000000000008e-06 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032245+0000", 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5619999999999997e-06 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032300+0000", 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4729999999999999e-05 2024-04-18T22:22:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032322+0000", 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2459e-05 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:3006 5.d 5:b5d83434:::benchmark_data_smithi032_22800_object3005:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.028643+0000", 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0049320550000000003, 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050356669999999997, 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3006 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028643+0000", 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028643+0000", 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028644+0000", 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028704+0000", 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0646999999999997e-05 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028712+0000", 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0530000000000007e-06 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028717+0000", 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3170000000000003e-06 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028769+0000", 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2611000000000002e-05 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028900+0000", 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013045099999999999 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.028982+0000", 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2509999999999994e-05 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030581+0000", 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001598506 2024-04-18T22:22:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.033624+0000", 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003043076 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.033673+0000", 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9425000000000002e-05 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:3013 5.d 5:bae871c2:::benchmark_data_smithi032_22800_object3012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.030718+0000", 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0028572570000000002, 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030696090000000001, 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3013 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030718+0000", 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030718+0000", 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030719+0000", 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.78e-07 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030918+0000", 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000199297 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030932+0000", 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3624e-05 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030939+0000", 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2099999999999996e-06 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031012+0000", 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3460000000000005e-05 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031159+0000", 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014642799999999999 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031272+0000", 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113439 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.033369+0000", 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020961560000000001 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:3014 5.b 5:d29aff3f:::benchmark_data_smithi032_22800_object3013:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.030955+0000", 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0026203860000000002, 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029260900000000001, 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3014 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030955+0000", 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030955+0000", 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030956+0000", 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5099999999999998e-07 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031029+0000", 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3415000000000001e-05 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031036+0000", 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9500000000000004e-06 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031042+0000", 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4550000000000003e-06 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031126+0000", 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4429000000000001e-05 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031198+0000", 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1888e-05 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031328+0000", 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130183 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032433+0000", 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011044970000000001 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:3015 5.8 5:1ab21164:::benchmark_data_smithi032_22800_object3014:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.031058+0000", 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0025176320000000001, 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029164949999999999, 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3015 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031058+0000", 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031058+0000", 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031058+0000", 2024-04-18T22:22:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8500000000000001e-07 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031087+0000", 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8741000000000001e-05 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031094+0000", 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1899999999999998e-06 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031100+0000", 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031186+0000", 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6451000000000003e-05 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031300+0000", 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113828 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031424+0000", 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012420899999999999 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032669+0000", 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012448069999999999 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4586.0:3010 5.a e22/19 5:5c251995:::benchmark_data_smithi032_22800_object3009:head v 22'193, mlcod=22'189)", 2024-04-18T22:22:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.030863+0000", 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.002712566, 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003197961, 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030863+0000", 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030863+0000", 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.030863+0000", 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.44e-07 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031173+0000", 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000310272 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031181+0000", 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9340000000000006e-06 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031187+0000", 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8850000000000004e-06 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031230+0000", 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2330999999999998e-05 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031247+0000", 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4586.0:3012 5.f e22/19 5:f26c1cdb:::benchmark_data_smithi032_22800_object3011:head v 22'186, mlcod=22'184)", 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.031310+0000", 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0022652950000000001, 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028192550000000001, 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:22:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031310+0000", 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031310+0000", 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031311+0000", 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4499999999999996e-07 2024-04-18T22:22:57.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031402+0000", 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1212999999999996e-05 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031428+0000", 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5741999999999998e-05 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031451+0000", 2024-04-18T22:22:57.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3665000000000001e-05 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031532+0000", 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0814000000000002e-05 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031557+0000", 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4647e-05 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:3018 5.e 5:766efea8:::benchmark_data_smithi032_22800_object3017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:57.031348+0000", 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0022271539999999999, 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002851148, 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3018 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031348+0000", 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031348+0000", 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031349+0000", 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000003e-07 2024-04-18T22:22:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031519+0000", 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016964200000000001 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031529+0000", 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031536+0000", 2024-04-18T22:22:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6170000000000001e-06 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031590+0000", 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3569000000000002e-05 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031698+0000", 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108179 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.031772+0000", 2024-04-18T22:22:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3579999999999997e-05 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:57.032921+0000", 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011498490000000001 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 8 2024-04-18T22:22:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:22:57.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:22:57.069 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:57.064+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:57.163 DEBUG:tasks.admin_socket:Testing objecter_requests with config {'test': 'http://git.ceph.com/?p={repo};a=blob_plain;f=src/test/admin_socket/objecter_requests;hb={branch}', 'branch': 'squid'} 2024-04-18T22:22:57.164 DEBUG:teuthology.orchestra.run.smithi032:> wget -q -O /home/ubuntu/cephtest/admin_socket_client.0/objecter_requests -- 'http://git.ceph.com/?p=ceph.git;a=blob_plain;f=src/test/admin_socket/objecter_requests;hb=squid' && chmod u=rx -- /home/ubuntu/cephtest/admin_socket_client.0/objecter_requests 2024-04-18T22:22:57.169 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:57.166+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:57.188 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 1 16 3607 3591 224.419 224.438 0.00677438 0.00444091 2024-04-18T22:22:57.266 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:22:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:22:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:22:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:22:57.267 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:22:57.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:57.268+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:57.292 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:22:57.371 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:57.371+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:57.472 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:57.470+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:57.492 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:22:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:22:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.377844432, 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:22:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375510540000001, 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:22:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:22:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:22:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:22:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375407640999999, 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:22:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:22:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:22:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:22:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375357973, 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:22:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375292298000002, 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:22:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:22:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:22:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:22:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:22:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:22:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375232551, 2024-04-18T22:22:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:22:57.529 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.529 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:22:57.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:22:57.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:22:57.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:22:57.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375152481000001, 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:22:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:22:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375091073, 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:22:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:22:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:22:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.375006760000002, 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:22:57.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.367851027, 2024-04-18T22:22:57.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:7 3.10 e17/15 3:0d82a743:::600.00000000:head v 17'2, mlcod=0'0)", 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124263+0000", 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.367597790000001, 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013138662000000001, 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124263+0000", 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124263+0000", 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124264+0000", 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7e-06 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124287+0000", 2024-04-18T22:22:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2543000000000001e-05 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125571+0000", 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001284589 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125594+0000", 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2619000000000001e-05 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126256+0000", 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066151800000000005 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126367+0000", 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011124900000000001 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136864+0000", 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010496745 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.137401+0000", 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053769899999999999 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:8 3.10 e17/15 3:09401a81:::601.00000000:head v 17'3, mlcod=0'0)", 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.125973+0000", 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.365887387000001, 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012744412, 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125973+0000", 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125973+0000", 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125974+0000", 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6300000000000004e-07 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125997+0000", 2024-04-18T22:22:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.315e-05 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126015+0000", 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8196e-05 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126029+0000", 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4134999999999999e-05 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128679+0000", 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026501770000000001 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128736+0000", 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6674e-05 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138644+0000", 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0099079389999999993 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138717+0000", 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3277999999999999e-05 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:1 3.14 e17/15 3:292cf221:::200.00000000:head v 17'3, mlcod=0'0)", 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.131768+0000", 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.3600919, 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011798101, 2024-04-18T22:22:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131768+0000", 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131768+0000", 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131770+0000", 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0470000000000001e-06 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131780+0000", 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0122999999999999e-05 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132233+0000", 2024-04-18T22:22:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045355900000000003 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132247+0000", 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3549e-05 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132823+0000", 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057586400000000004 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132880+0000", 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7141000000000001e-05 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143482+0000", 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010602395000000001 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143567+0000", 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4423000000000001e-05 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:3 3.14 e17/15 3:287034c4:::1.00000000.inode:head v 17'4, mlcod=0'0)", 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.132539+0000", 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.359321616999999, 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013238735999999999, 2024-04-18T22:22:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132539+0000", 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132539+0000", 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132540+0000", 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132549+0000", 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9870000000000005e-06 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132565+0000", 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6452000000000001e-05 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132582+0000", 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7433000000000001e-05 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133613+0000", 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010309729999999999 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133666+0000", 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2335e-05 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.144787+0000", 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011121341 2024-04-18T22:22:57.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.145777+0000", 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0009903449999999999 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "pg_update_log_missing(3.14 epoch 17/15 rep_tid 172 entries 17'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:24 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.133685+0000", 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.358174871999999, 2024-04-18T22:22:57.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012132565, 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133685+0000", 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133685+0000", 2024-04-18T22:22:57.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133686+0000", 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-07 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133700+0000", 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.416e-05 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133734+0000", 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3312000000000002e-05 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133768+0000", 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4304999999999998e-05 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135311+0000", 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001542823 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.145818+0000", 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010507215 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:4 5.e 5:7200c3d5:::benchmark_data_smithi032_22800_object3:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.192526+0000", 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2993341279999999, 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011925418, 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.192526+0000", 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.192526+0000", 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.192535+0000", 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9900000000000003e-06 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193024+0000", 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000488493 2024-04-18T22:22:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193098+0000", 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193122+0000", 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3580000000000001e-05 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193291+0000", 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000169486 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193307+0000", 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6039e-05 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200301+0000", 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069935190000000001 2024-04-18T22:22:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200603+0000", 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030201000000000002 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200930+0000", 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032707800000000001 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.203937+0000", 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030073270000000002 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204317+0000", 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038001799999999998 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204391+0000", 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3799000000000003e-05 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204452+0000", 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0528000000000001e-05 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:5 5.d 5:bb067dfb:::benchmark_data_smithi032_22800_object4:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.193181+0000", 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.298679656, 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011674013, 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193181+0000", 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193181+0000", 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193182+0000", 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.643e-06 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193331+0000", 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000148954 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193355+0000", 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3866000000000002e-05 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193378+0000", 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2623e-05 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193543+0000", 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000164859 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193558+0000", 2024-04-18T22:22:57.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5477000000000001e-05 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.199742+0000", 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061841600000000002 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200030+0000", 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028814999999999998 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200355+0000", 2024-04-18T22:22:57.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032479299999999998 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.202301+0000", 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019453179999999999 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204644+0000", 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023438690000000002 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204765+0000", 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012050299999999999 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204855+0000", 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9797999999999996e-05 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:13 5.e 5:7b9552ac:::benchmark_data_smithi032_22800_object12:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.193668+0000", 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2981920140000001, 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012548224, 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 13 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193668+0000", 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193668+0000", 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193671+0000", 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1600000000000001e-06 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193811+0000", 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014051100000000001 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193833+0000", 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2158999999999999e-05 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193851+0000", 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999998e-05 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193979+0000", 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000128417 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:22:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193988+0000", 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6729999999999998e-06 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.201843+0000", 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0078549350000000004 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.202017+0000", 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017436900000000001 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.202218+0000", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020010500000000001 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204589+0000", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023712680000000002 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.205968+0000", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001379405 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.206103+0000", 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013432 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.206217+0000", 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114042 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:124 5.e 5:73ef8cd7:::benchmark_data_smithi032_22800_object123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.245181+0000", 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2466797190000001, 2024-04-18T22:22:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011318101000000001, 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 124 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245181+0000", 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245181+0000", 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245182+0000", 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.75e-06 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245300+0000", 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011746700000000001 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245325+0000", 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5367e-05 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245343+0000", 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7672e-05 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245477+0000", 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133874 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245784+0000", 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030675599999999999 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.246139+0000", 2024-04-18T22:22:57.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035594499999999998 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.247886+0000", 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001746339 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256319+0000", 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084334519999999993 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256408+0000", 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8857999999999995e-05 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256499+0000", 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0620999999999999e-05 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:125 5.b 5:d87693cd:::benchmark_data_smithi032_22800_object124:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.245386+0000", 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.246474337, 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011451964, 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 125 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:22:57.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245386+0000", 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245386+0000", 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245388+0000", 2024-04-18T22:22:57.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7099999999999999e-06 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245520+0000", 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132087 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245562+0000", 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1755999999999998e-05 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245579+0000", 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7538999999999999e-05 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245683+0000", 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103701 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245985+0000", 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030188699999999998 2024-04-18T22:22:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.246354+0000", 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000369584 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.248851+0000", 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024970970000000002 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256649+0000", 2024-04-18T22:22:57.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077976360000000002 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256753+0000", 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010364700000000001 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256838+0000", 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5320000000000003e-05 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:22:57.614 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:22:57.614 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:22:57.615 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:22:57.616 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:57.612+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:57.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --admin-daemon /var/run/ceph/ceph-client.0.asok objecter_requests 2024-04-18T22:22:57.715 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:57.715+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:57.774 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:22:57.774 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:22:57.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.774 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5925 5.2 5:4e6272fb:::benchmark_data_smithi032_22800_object5924:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.771210+0000", 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0036705560000000002, 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037079610000000001, 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5925 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771210+0000", 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771210+0000", 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771211+0000", 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2699999999999998e-07 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771277+0000", 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5885999999999999e-05 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771288+0000", 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0944999999999999e-05 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771293+0000", 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3990000000000003e-06 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771337+0000", 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3525000000000001e-05 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771454+0000", 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011681299999999999 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771595+0000", 2024-04-18T22:22:57.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014111000000000001 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774334+0000", 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027388619999999999 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5926 5.7 5:e3e4945e:::benchmark_data_smithi032_22800_object5925:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.771307+0000", 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0035739209999999999, 2024-04-18T22:22:57.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003719277, 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5926 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771307+0000", 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771307+0000", 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771307+0000", 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3599999999999999e-07 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771374+0000", 2024-04-18T22:22:57.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6845000000000001e-05 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771380+0000", 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6239999999999999e-06 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771384+0000", 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7570000000000002e-06 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771458+0000", 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4739000000000004e-05 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771637+0000", 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017824799999999999 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771805+0000", 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016865200000000001 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:57.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774328+0000", 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025232850000000001 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5929 5.2 5:435d6dd5:::benchmark_data_smithi032_22800_object5928:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.771523+0000", 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033578229999999998, 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036097490000000002, 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.784 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5929 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771523+0000", 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771523+0000", 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771523+0000", 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7300000000000002e-07 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771559+0000", 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5326e-05 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771563+0000", 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7040000000000002e-06 2024-04-18T22:22:57.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.771567+0000", 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3680000000000002e-06 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.772153+0000", 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000586142 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.772345+0000", 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000191764 2024-04-18T22:22:57.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.772492+0000", 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000146885 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774340+0000", 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018483530000000001 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5931 5.1 5:8c29d535:::benchmark_data_smithi032_22800_object5930:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.773077+0000", 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018038559999999999, 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021417150000000002, 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5931 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.789 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773077+0000", 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773077+0000", 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773078+0000", 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6700000000000003e-07 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773148+0000", 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0362000000000002e-05 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773158+0000", 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0387000000000001e-05 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773164+0000", 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9050000000000002e-06 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773221+0000", 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6339999999999999e-05 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773369+0000", 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014791699999999999 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773536+0000", 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000167386 2024-04-18T22:22:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:5932 5.6 e22/19 5:66607004:::benchmark_data_smithi032_22800_object5931:head v 22'400, mlcod=22'398)", 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.773624+0000", 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001256959, 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001659226, 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773624+0000", 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773624+0000", 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773624+0000", 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7899999999999999e-07 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773662+0000", 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7187000000000002e-05 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773670+0000", 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5569999999999996e-06 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773676+0000", 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7189999999999998e-06 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773728+0000", 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2482999999999999e-05 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.773742+0000", 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.418e-05 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5934 5.1 5:89d6f2a7:::benchmark_data_smithi032_22800_object5933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.774061+0000", 2024-04-18T22:22:57.796 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00082029900000000003, 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012688210000000001, 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5934 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774061+0000", 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774061+0000", 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774062+0000", 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2109999999999999e-06 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774166+0000", 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103798 2024-04-18T22:22:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774177+0000", 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1168999999999999e-05 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774188+0000", 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.084e-05 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774334+0000", 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014629500000000001 2024-04-18T22:22:57.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774414+0000", 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0016000000000001e-05 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774499+0000", 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4864999999999995e-05 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5935 5.9 5:9d4e5582:::benchmark_data_smithi032_22800_object5934:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.774413+0000", 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00046775199999999999, 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00100215, 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5935 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774413+0000", 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774413+0000", 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774414+0000", 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9699999999999995e-07 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774481+0000", 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7021999999999996e-05 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774489+0000", 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0150000000000003e-06 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774494+0000", 2024-04-18T22:22:57.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3430000000000003e-06 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774538+0000", 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3798999999999999e-05 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774702+0000", 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016363900000000001 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:57.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774815+0000", 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011294399999999999 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5938 5.f 5:f90f27bc:::benchmark_data_smithi032_22800_object5937:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.774641+0000", 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00023982800000000001, 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00085545100000000002, 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.805 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5938 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774641+0000", 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774641+0000", 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774642+0000", 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0999999999999998e-07 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774723+0000", 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1688999999999997e-05 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774731+0000", 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7149999999999998e-06 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774736+0000", 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4959999999999998e-06 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774785+0000", 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9425000000000002e-05 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774914+0000", 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012878600000000001 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775067+0000", 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015316700000000001 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:5940 5.1 5:8a81dc09:::benchmark_data_smithi032_22800_object5939:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.774959+0000", 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9999213, 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00062354700000000001, 2024-04-18T22:22:57.809 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 5940 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774959+0000", 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774959+0000", 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.774960+0000", 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.16e-07 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775020+0000", 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0299e-05 2024-04-18T22:22:57.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775040+0000", 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9765000000000001e-05 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775047+0000", 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5060000000000001e-06 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775097+0000", 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0010999999999999e-05 2024-04-18T22:22:57.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775192+0000", 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5668999999999995e-05 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775265+0000", 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2854000000000004e-05 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:5937 5.3 e22/19 5:cbe78781:::benchmark_data_smithi032_22800_object5936:head v 22'344, mlcod=22'342)", 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:57.775226+0000", 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9996548, 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00044148499999999998, 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775226+0000", 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775226+0000", 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775226+0000", 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2899999999999999e-07 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775279+0000", 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2935000000000001e-05 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775288+0000", 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7669999999999992e-06 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775293+0000", 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5750000000000002e-06 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775345+0000", 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2632e-05 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:22:57.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:57.775363+0000", 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7736000000000001e-05 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 10 2024-04-18T22:22:57.817 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:22:57.818 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:57.814+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:57.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6077, 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.526fa866", 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6076", 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6076", 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.820 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.818846+0000", 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6071, 2024-04-18T22:22:57.821 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.e68054e4", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6070", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6070", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.816552+0000", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6074, 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.97c04384", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6073", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6073", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.823 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.816719+0000", 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6075, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.3f1b0b60", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6074", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6074", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.825 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.816752+0000", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6076, 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.4a8f342b", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6075", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6075", 2024-04-18T22:22:57.826 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.817137+0000", 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6079, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.a2d552e8", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6078", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6078", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.828 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819247+0000", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6080, 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.fca792db", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6079", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6079", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.829 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819313+0000", 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6081, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.92a75d8b", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6080", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6080", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.831 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819331+0000", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6083, 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.81f665b", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6082", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6082", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.832 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.907366s", 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0, 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819523+0000", 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6084, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.f982056e", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6083", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6083", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.907366s", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.834 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819544+0000", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6085, 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.6d4599e0", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6084", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6084", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.835 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.907366s", 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0, 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819553+0000", 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6086, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.4237efcb", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6085", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6085", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.907366s", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.837 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.820631+0000", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6068, 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.a6e10841", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6067", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6067", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.838 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.899366s", 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0079996730000000005, 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.814123+0000", 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6069, 2024-04-18T22:22:57.839 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.b30a0cb2", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6068", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6068", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.816406+0000", 2024-04-18T22:22:57.840 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6073, 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.ebb446b1", 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6072", 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6072", 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.816705+0000", 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6082, 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg": "5.ac6b75c7", 2024-04-18T22:22:57.842 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "object_id": "benchmark_data_smithi032_22800_object6081", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "object_locator": "@5", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_id": "benchmark_data_smithi032_22800_object6081", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "target_object_locator": "@5", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "paused": 0, 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "used_replica": 0, 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "precalc_pgid": 0, 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_sent": "709.903366s", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0039998359999999997, 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "attempts": 1, 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "snapid": "head", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_context": "0=[]", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "mtime": "2024-04-18T22:22:57.819394+0000", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ops": [ 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2024-04-18T22:22:57.843 INFO:teuthology.orchestra.run.smithi032.stdout: "write 0~65536 in=65536b" 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_ops": [], 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ops": [], 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_stat_ops": [], 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_ops": [], 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout: "command_ops": [] 2024-04-18T22:22:57.844 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:22:57.845 DEBUG:tasks.admin_socket:admin socket command objecter_requests returned { "ops": [ { "tid": 6077, "pg": "5.526fa866", "osd": 0, "object_id": "benchmark_data_smithi032_22800_object6076", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6076", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.818846+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6071, "pg": "5.e68054e4", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6070", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6070", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.816552+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6074, "pg": "5.97c04384", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6073", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6073", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.816719+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6075, "pg": "5.3f1b0b60", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6074", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6074", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.816752+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6076, "pg": "5.4a8f342b", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6075", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6075", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.817137+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6079, "pg": "5.a2d552e8", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6078", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6078", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819247+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6080, "pg": "5.fca792db", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6079", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6079", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819313+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6081, "pg": "5.92a75d8b", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6080", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6080", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819331+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6083, "pg": "5.81f665b", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6082", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6082", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.907366s", "age": 0, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819523+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6084, "pg": "5.f982056e", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6083", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6083", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.907366s", "age": 0, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819544+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6085, "pg": "5.6d4599e0", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6084", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6084", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.907366s", "age": 0, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819553+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6086, "pg": "5.4237efcb", "osd": 1, "object_id": "benchmark_data_smithi032_22800_object6085", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6085", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.907366s", "age": 0, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.820631+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6068, "pg": "5.a6e10841", "osd": 2, "object_id": "benchmark_data_smithi032_22800_object6067", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6067", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.899366s", "age": 0.0079996730000000005, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.814123+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6069, "pg": "5.b30a0cb2", "osd": 2, "object_id": "benchmark_data_smithi032_22800_object6068", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6068", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.816406+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6073, "pg": "5.ebb446b1", "osd": 2, "object_id": "benchmark_data_smithi032_22800_object6072", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6072", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.816705+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 6082, "pg": "5.ac6b75c7", "osd": 2, "object_id": "benchmark_data_smithi032_22800_object6081", "object_locator": "@5", "target_object_id": "benchmark_data_smithi032_22800_object6081", "target_object_locator": "@5", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "709.903366s", "age": 0.0039998359999999997, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2024-04-18T22:22:57.819394+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] } ], "linger_ops": [], "pool_ops": [], "pool_stat_ops": [], "statfs_ops": [], "command_ops": [] } 2024-04-18T22:22:57.846 DEBUG:teuthology.orchestra.run.smithi032:> /home/ubuntu/cephtest/admin_socket_client.0/objecter_requests 2024-04-18T22:22:57.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:22:57.876 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/admin_socket_client.0 2024-04-18T22:22:57.919 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:57.918+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:57.926 DEBUG:teuthology.parallel:result is None 2024-04-18T22:22:57.926 DEBUG:teuthology.run_tasks:Unwinding manager radosbench 2024-04-18T22:22:57.938 INFO:tasks.radosbench:joining radosbench (timing out after 2100s) 2024-04-18T22:22:57.939 INFO:teuthology.orchestra.run:waiting for 2100 2024-04-18T22:22:58.006 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:22:58.007 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:22:58.007 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:22:58.007 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:22:58.007 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:22:58.021 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:58.018+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:58.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:22:58.086 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:22:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:22:58.106 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:22:58.107 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:22:58.121 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:58.118+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:58.126 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:22:58.126 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:22:58.175 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:58.175 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:58.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 2 16 7419 7403 231.322 238.25 0.00229869 0.00431382 2024-04-18T22:22:58.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:22:58.221 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:58.216+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:58.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:58.320+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:58.369 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:58.369 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:58.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:22:58.433 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:58.435+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:58.524 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:58.522+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:58.544 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:58.544 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:58.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:22:58.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:22:58.590 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:22:58.590 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-18T22:22:58.590 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-04-18T22:22:58.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-04-18T22:22:58.653 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:58.655+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:58.725 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:58.722+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:58.762 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:58.762 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:58.784 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:22:58.829 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:58.831+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:58.926 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:58.922+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:58.940 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:58.940 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:58.958 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:22:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.2 primary-affinity to 0 (802) 2024-04-18T22:22:58.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:22:59.026 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:59.027+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:59.115 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:59.115 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:59.127 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:59.127+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:59.129 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:22:59.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 3 16 9829 9813 204.417 150.625 0.00244445 0.00454866 2024-04-18T22:22:59.229 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:59.227+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:59.251 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:59.251 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:59.265 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:22:59.330 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:59.331+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:59.397 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:59.397 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:59.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:22:59.458 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:59.456+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:59.527 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:59.527 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:59.531 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:59.528+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:59.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:22:59.633 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:59.634+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:59.704 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:59.704 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:59.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:22:59.734 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:22:59.732+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:22:59.835 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:22:59.835+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:22:59.848 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:22:59.848 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:22:59.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:22:59.936 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:22:59.934+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:22:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:23:00.037 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:00.039+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:00.048 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:00.048 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:00.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:00.137 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:00.134+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:00.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:00.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 4 16 10328 10312 161.109 31.1875 0.00659594 0.00619652 2024-04-18T22:23:00.239 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:00.239+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:00.273 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:00.273 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:00.296 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:00.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:00.312 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:00.329 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:00.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:00.336+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:00.351 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:00.441 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:00.438+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:00.454 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:00.454 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:00.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.454 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:11088 5.e 5:7efff323:::benchmark_data_smithi032_22800_object11087:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:00.454119+0000", 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0019762030000000002, 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020011180000000001, 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11088 2024-04-18T22:23:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454119+0000", 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454119+0000", 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454120+0000", 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1689999999999999e-06 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454207+0000", 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6446999999999994e-05 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454221+0000", 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3786e-05 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454229+0000", 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0250000000000002e-06 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454267+0000", 2024-04-18T22:23:00.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8621999999999997e-05 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454355+0000", 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7282999999999998e-05 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.454436+0000", 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1500000000000002e-05 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455386+0000", 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00094962900000000001 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.460 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:11090 5.d 5:b237fee6:::benchmark_data_smithi032_22800_object11089:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:00.455033+0000", 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0010624810000000001, 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011780860000000001, 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11090 2024-04-18T22:23:00.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455033+0000", 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455033+0000", 2024-04-18T22:23:00.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455034+0000", 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455102+0000", 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8328e-05 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455112+0000", 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7669999999999998e-06 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455117+0000", 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1220000000000003e-06 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455164+0000", 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6708000000000001e-05 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455258+0000", 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3951999999999998e-05 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.455352+0000", 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4634000000000002e-05 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:11095 5.0 5:0d369a24:::benchmark_data_smithi032_22800_object11094:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:00.456009+0000", 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.6718999999999994e-05, 2024-04-18T22:23:00.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028624500000000002, 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11095 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456009+0000", 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456009+0000", 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456010+0000", 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0449999999999999e-06 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456108+0000", 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8270999999999995e-05 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456122+0000", 2024-04-18T22:23:00.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4146e-05 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456129+0000", 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6749999999999996e-06 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456185+0000", 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450999999999999e-05 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:11096 5.0 5.cca1f400 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:00.456146+0000", 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9999495, 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021004300000000001, 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11096 2024-04-18T22:23:00.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456146+0000", 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456146+0000", 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456147+0000", 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8700000000000005e-07 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456208+0000", 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0825999999999997e-05 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456216+0000", 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3990000000000004e-06 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:00.456222+0000", 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0079999999999999e-06 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2024-04-18T22:23:00.474 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:00.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:00.542 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:00.543+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:00.642 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:00.643+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:00.662 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:00.662 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:00.684 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:00.743 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:00.742+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:00.835 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:00.835 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:23:00.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.721780921000001, 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:23:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:23:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:23:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.719447029000001, 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:23:00.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:23:00.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:23:00.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.71934413, 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:23:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:23:00.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.719294462000001, 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.853 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.719228786999999, 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:23:00.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:00.864 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.719169040000001, 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:00.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:23:00.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.719088970000001, 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:23:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:23:00.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.719027562000001, 2024-04-18T22:23:00.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.718943248999999, 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:00.880 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:23:00.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.711787516000001, 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:23:00.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.888 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:7 3.10 e17/15 3:0d82a743:::600.00000000:head v 17'2, mlcod=0'0)", 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124263+0000", 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.711534278999999, 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013138662000000001, 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124263+0000", 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124263+0000", 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124264+0000", 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7e-06 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124287+0000", 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2543000000000001e-05 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125571+0000", 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001284589 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125594+0000", 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2619000000000001e-05 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126256+0000", 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066151800000000005 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126367+0000", 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011124900000000001 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136864+0000", 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010496745 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.137401+0000", 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053769899999999999 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:8 3.10 e17/15 3:09401a81:::601.00000000:head v 17'3, mlcod=0'0)", 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.125973+0000", 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.709823876000002, 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012744412, 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125973+0000", 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125973+0000", 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125974+0000", 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6300000000000004e-07 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125997+0000", 2024-04-18T22:23:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.315e-05 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126015+0000", 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8196e-05 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126029+0000", 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4134999999999999e-05 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128679+0000", 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026501770000000001 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128736+0000", 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6674e-05 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138644+0000", 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0099079389999999993 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138717+0000", 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3277999999999999e-05 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:1 3.14 e17/15 3:292cf221:::200.00000000:head v 17'3, mlcod=0'0)", 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.131768+0000", 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.704028389000001, 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011798101, 2024-04-18T22:23:00.897 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131768+0000", 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131768+0000", 2024-04-18T22:23:00.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131770+0000", 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0470000000000001e-06 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131780+0000", 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0122999999999999e-05 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132233+0000", 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045355900000000003 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132247+0000", 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3549e-05 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132823+0000", 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057586400000000004 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132880+0000", 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7141000000000001e-05 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143482+0000", 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010602395000000001 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143567+0000", 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4423000000000001e-05 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:3 3.14 e17/15 3:287034c4:::1.00000000.inode:head v 17'4, mlcod=0'0)", 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.132539+0000", 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.703258106, 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013238735999999999, 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132539+0000", 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132539+0000", 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132540+0000", 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132549+0000", 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9870000000000005e-06 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132565+0000", 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6452000000000001e-05 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132582+0000", 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7433000000000001e-05 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133613+0000", 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010309729999999999 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133666+0000", 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2335e-05 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.144787+0000", 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011121341 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.145777+0000", 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0009903449999999999 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "pg_update_log_missing(3.14 epoch 17/15 rep_tid 172 entries 17'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:24 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-04-18T22:23:00.906 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.133685+0000", 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.702111361, 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012132565, 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133685+0000", 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133685+0000", 2024-04-18T22:23:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133686+0000", 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5000000000000002e-07 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133700+0000", 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.416e-05 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133734+0000", 2024-04-18T22:23:00.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3312000000000002e-05 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133768+0000", 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4304999999999998e-05 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135311+0000", 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001542823 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.145818+0000", 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010507215 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:4 5.e 5:7200c3d5:::benchmark_data_smithi032_22800_object3:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.192526+0000", 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.6432706169999998, 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011925418, 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.192526+0000", 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.192526+0000", 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.192535+0000", 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9900000000000003e-06 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193024+0000", 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000488493 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193098+0000", 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4550999999999998e-05 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193122+0000", 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3580000000000001e-05 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193291+0000", 2024-04-18T22:23:00.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000169486 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193307+0000", 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6039e-05 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200301+0000", 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069935190000000001 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200603+0000", 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030201000000000002 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200930+0000", 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032707800000000001 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.203937+0000", 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030073270000000002 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204317+0000", 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038001799999999998 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204391+0000", 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3799000000000003e-05 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204452+0000", 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0528000000000001e-05 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:5 5.d 5:bb067dfb:::benchmark_data_smithi032_22800_object4:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.193181+0000", 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.6426161449999999, 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011674013, 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193181+0000", 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193181+0000", 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193182+0000", 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.643e-06 2024-04-18T22:23:00.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193331+0000", 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000148954 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193355+0000", 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3866000000000002e-05 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193378+0000", 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2623e-05 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193543+0000", 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000164859 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193558+0000", 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5477000000000001e-05 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.199742+0000", 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061841600000000002 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200030+0000", 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028814999999999998 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.200355+0000", 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032479299999999998 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.202301+0000", 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019453179999999999 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204644+0000", 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023438690000000002 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204765+0000", 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012050299999999999 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204855+0000", 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9797999999999996e-05 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:13 5.e 5:7b9552ac:::benchmark_data_smithi032_22800_object12:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.193668+0000", 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.6421285030000004, 2024-04-18T22:23:00.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012548224, 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 13 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193668+0000", 2024-04-18T22:23:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193668+0000", 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193671+0000", 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1600000000000001e-06 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193811+0000", 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014051100000000001 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193833+0000", 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2158999999999999e-05 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193851+0000", 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999998e-05 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193979+0000", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000128417 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.193988+0000", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6729999999999998e-06 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.201843+0000", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0078549350000000004 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.202017+0000", 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017436900000000001 2024-04-18T22:23:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.202218+0000", 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020010500000000001 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.204589+0000", 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023712680000000002 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.205968+0000", 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001379405 2024-04-18T22:23:00.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.206103+0000", 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013432 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.206217+0000", 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114042 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:124 5.e 5:73ef8cd7:::benchmark_data_smithi032_22800_object123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.245181+0000", 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5906162080000001, 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011318101000000001, 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 124 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245181+0000", 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245181+0000", 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245182+0000", 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.75e-06 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245300+0000", 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011746700000000001 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245325+0000", 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5367e-05 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245343+0000", 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7672e-05 2024-04-18T22:23:00.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245477+0000", 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133874 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245784+0000", 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030675599999999999 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.246139+0000", 2024-04-18T22:23:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035594499999999998 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.247886+0000", 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001746339 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256319+0000", 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084334519999999993 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256408+0000", 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8857999999999995e-05 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256499+0000", 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0620999999999999e-05 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:125 5.b 5:d87693cd:::benchmark_data_smithi032_22800_object124:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:56.245386+0000", 2024-04-18T22:23:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5904108260000003, 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011451964, 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 125 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245386+0000", 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245386+0000", 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245388+0000", 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7099999999999999e-06 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245520+0000", 2024-04-18T22:23:00.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132087 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245562+0000", 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1755999999999998e-05 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245579+0000", 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7538999999999999e-05 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:00.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245683+0000", 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103701 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.245985+0000", 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030188699999999998 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.246354+0000", 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000369584 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:00.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.248851+0000", 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024970970000000002 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256649+0000", 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077976360000000002 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256753+0000", 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010364700000000001 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:00.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:56.256838+0000", 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5320000000000003e-05 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:00.942 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:00.944 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:00.944 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:00.942+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:01.044 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:01.042+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:01.104 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12893 5.d e24/19 5:b8c95b49:::benchmark_data_smithi032_22800_object12892:head v 24'783, mlcod=24'780)", 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.101132+0000", 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0046667089999999998, 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046837340000000002, 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101132+0000", 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101132+0000", 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101133+0000", 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.122e-06 2024-04-18T22:23:01.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101199+0000", 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5438999999999995e-05 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101217+0000", 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7920000000000001e-05 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101229+0000", 2024-04-18T22:23:01.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2625e-05 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101299+0000", 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9321999999999998e-05 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101325+0000", 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5904000000000001e-05 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.105769+0000", 2024-04-18T22:23:01.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044442420000000002 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12896 5.2 e24/23 5:4966f5ab:::benchmark_data_smithi032_22800_object12895:head v 24'833, mlcod=24'831)", 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.101874+0000", 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0039254279999999999, 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039847559999999999, 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101874+0000", 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101874+0000", 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101875+0000", 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.108e-06 2024-04-18T22:23:01.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101962+0000", 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7415000000000002e-05 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101981+0000", 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.933e-05 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.101993+0000", 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1995e-05 2024-04-18T22:23:01.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102066+0000", 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2825999999999997e-05 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102086+0000", 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9493000000000001e-05 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.112 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12897 5.3 e24/19 5:c343a577:::benchmark_data_smithi032_22800_object12896:head v 24'835, mlcod=24'833)", 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.102822+0000", 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0029773759999999999, 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003068786, 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102822+0000", 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102822+0000", 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102822+0000", 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1999999999999998e-07 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102897+0000", 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4901999999999995e-05 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.114 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102911+0000", 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3417e-05 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102918+0000", 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.396e-06 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.102979+0000", 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0958000000000001e-05 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103006+0000", 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6747999999999999e-05 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12900 5.c e24/19 5:3032bc0e:::benchmark_data_smithi032_22800_object12899:head v 24'850, mlcod=24'848)", 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.103285+0000", 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002513562, 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002635484, 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.116 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103285+0000", 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103285+0000", 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103286+0000", 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3200000000000003e-07 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103385+0000", 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8774e-05 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103408+0000", 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3014e-05 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103414+0000", 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3389999999999998e-06 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103479+0000", 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4783999999999995e-05 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103504+0000", 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4346e-05 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.119 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12898 5.d e24/19 5:b3ddfb4f:::benchmark_data_smithi032_22800_object12897:head v 24'784, mlcod=24'780)", 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.103263+0000", 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0025356630000000001, 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026952389999999999, 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103263+0000", 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103263+0000", 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103264+0000", 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0699999999999996e-07 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103417+0000", 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015345300000000001 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103429+0000", 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1769e-05 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103439+0000", 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8379999999999997e-06 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103494+0000", 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4654000000000002e-05 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103515+0000", 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1761000000000001e-05 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12899 5.f e24/23 5:f4fd9075:::benchmark_data_smithi032_22800_object12898:head v 24'748, mlcod=24'744)", 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.103464+0000", 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002334437, 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025502620000000002, 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103464+0000", 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103464+0000", 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103465+0000", 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0100000000000005e-07 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103511+0000", 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6322000000000001e-05 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103526+0000", 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5e-05 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103534+0000", 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2130000000000002e-06 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103604+0000", 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0203999999999994e-05 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103637+0000", 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3345e-05 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12901 5.1 e24/23 5:8468fbc5:::benchmark_data_smithi032_22800_object12900:head v 24'784, mlcod=24'782)", 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.103979+0000", 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001819701, 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020959490000000002, 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.127 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103979+0000", 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103979+0000", 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.103980+0000", 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.049e-06 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104045+0000", 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5189000000000002e-05 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104064+0000", 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9012999999999999e-05 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104074+0000", 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7929999999999992e-06 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104143+0000", 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8297e-05 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104169+0000", 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6695000000000001e-05 2024-04-18T22:23:01.130 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12902 5.5 e24/19 5:a5f4c007:::benchmark_data_smithi032_22800_object12901:head v 24'753, mlcod=24'750)", 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.104912+0000", 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00088737199999999999, 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012098, 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.131 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104912+0000", 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104912+0000", 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104934+0000", 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.283e-05 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104978+0000", 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3959999999999999e-05 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104991+0000", 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2245e-05 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.104998+0000", 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0079999999999996e-06 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.105065+0000", 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6933000000000004e-05 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.105089+0000", 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4204000000000001e-05 2024-04-18T22:23:01.134 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12903 5.2 e24/23 5:4307d5fb:::benchmark_data_smithi032_22800_object12902:head v 24'834, mlcod=24'831)", 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.105997+0000", 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9998021, 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000180034, 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.105997+0000", 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.105997+0000", 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.105998+0000", 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4199999999999995e-07 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106031+0000", 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3701999999999997e-05 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106041+0000", 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0006999999999999e-05 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106050+0000", 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0480000000000004e-06 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106105+0000", 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4552000000000002e-05 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106128+0000", 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3023999999999998e-05 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:12904 5.3 e24/19)", 2024-04-18T22:23:01.138 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:01.106146+0000", 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9996533, 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8953e-05, 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106146+0000", 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106146+0000", 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106146+0000", 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1099999999999995e-07 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106179+0000", 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2838e-05 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106187+0000", 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2160000000000006e-06 2024-04-18T22:23:01.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:01.106195+0000", 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9640000000000003e-06 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 10 2024-04-18T22:23:01.141 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:01.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:01.145 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:01.147+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:01.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 5 16 13145 13129 164.096 176.062 0.00369193 0.00608609 2024-04-18T22:23:01.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:01.244+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:01.289 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:01.289 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:01.289 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:01.289 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:01.289 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:01.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:01.346 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:01.342+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:01.447 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:01.447+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:01.492 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4144.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e10)", 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.029265416000001, 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025591307000000001, 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.493 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4144", 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1049496302", 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 141 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:23:01.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461622+0000", 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9400000000000004e-07 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461627+0000", 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6559999999999997e-06 2024-04-18T22:23:01.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461638+0000", 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0594000000000001e-05 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461645+0000", 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2099999999999996e-06 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480273+0000", 2024-04-18T22:23:01.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018627812000000001 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480507+0000", 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00023432800000000001 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480828+0000", 2024-04-18T22:23:01.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032085799999999998 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.482445+0000", 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001617081 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487064+0000", 2024-04-18T22:23:01.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046189179999999996 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487149+0000", 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4575999999999998e-05 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487212+0000", 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3479999999999995e-05 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.499 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:6 3.10 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375929593999999, 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027601216000000001, 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114960+0000", 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.88e-06 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114981+0000", 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0877999999999999e-05 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115008+0000", 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7617999999999999e-05 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115031+0000", 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2302000000000001e-05 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121370+0000", 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063390540000000002 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121632+0000", 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000262539 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121921+0000", 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028841099999999997 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135317+0000", 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013396105 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142295+0000", 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0069786539999999999 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142445+0000", 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000149968 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142558+0000", 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011280700000000001 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.505 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375817033000001, 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:01.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375748434999998, 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375652589000001, 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.516 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.521 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375595706999999, 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:01.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:01.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:01.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:01.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.527 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375536035, 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:01.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:01.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.532 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375478073, 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:01.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:15 3.14 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:23:01.539 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375413455, 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027152815, 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115474+0000", 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8700000000000004e-07 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115482+0000", 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2560000000000002e-06 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115500+0000", 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8224999999999999e-05 2024-04-18T22:23:01.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115514+0000", 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3648999999999999e-05 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115656+0000", 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000141865 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115959+0000", 2024-04-18T22:23:01.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030357600000000001 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116299+0000", 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033980600000000001 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126282+0000", 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0099824879999999994 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142386+0000", 2024-04-18T22:23:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016104535999999999 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142517+0000", 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013121000000000001 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142626+0000", 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108317 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:16 3.1c 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375321919000001, 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027983006000000001, 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115566+0000", 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.127e-06 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115573+0000", 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0559999999999999e-06 2024-04-18T22:23:01.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115591+0000", 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8615e-05 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115608+0000", 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6815000000000001e-05 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117673+0000", 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020651689999999999 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.118082+0000", 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040828099999999998 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.118459+0000", 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037744699999999999 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126370+0000", 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079108990000000007 2024-04-18T22:23:01.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143152+0000", 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016782488000000002 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143434+0000", 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000281988 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143548+0000", 2024-04-18T22:23:01.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113121 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375258380999998, 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:01.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375201451999999, 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.555 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:01.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:01.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375105565999998, 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:01.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:01.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:01.565 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.375043238, 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.37497076, 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.571 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:2 3.f 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.576 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.374910528000001, 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026491475, 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.06e-06 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115983+0000", 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2820000000000002e-06 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116005+0000", 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1525000000000001e-05 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116018+0000", 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3372e-05 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116290+0000", 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000271453 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116676+0000", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038682899999999999 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117663+0000", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00098642700000000001 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125645+0000", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079826340000000006 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142118+0000", 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016472647 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142261+0000", 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014285799999999999 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142467+0000", 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000206388 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.581 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.374818292, 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:01.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:01.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:01.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:01.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.374739568999999, 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:01.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:01.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:01.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.373218713, 2024-04-18T22:23:01.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:01.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:01.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:01.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.597 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.373120361000002, 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:01.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:01.601 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:01.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:01.604+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:01.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:01.703 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:01.702+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:01.761 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14666 5.a 5:509608ae:::benchmark_data_smithi032_22800_object14665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.756180+0000", 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0075604030000000003, 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075847659999999997, 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.762 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14666 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756180+0000", 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756180+0000", 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756181+0000", 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0699999999999999e-06 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756245+0000", 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3689999999999995e-05 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756257+0000", 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.238e-05 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756266+0000", 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8759999999999994e-06 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756332+0000", 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6353000000000007e-05 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756435+0000", 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010283799999999999 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.756517+0000", 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1967999999999999e-05 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760044+0000", 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035266540000000002 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.766 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14672 5.a 5:5155d664:::benchmark_data_smithi032_22800_object14671:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.758578+0000", 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0051623600000000004, 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052384889999999998, 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14672 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.758578+0000", 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.758578+0000", 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.758579+0000", 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1299999999999999e-07 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.758617+0000", 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7858000000000002e-05 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.758624+0000", 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3250000000000002e-06 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.758629+0000", 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8489999999999997e-06 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759059+0000", 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00043019199999999998 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759199+0000", 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000139956 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759410+0000", 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000211225 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761964+0000", 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025540659999999998 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14674 5.a 5:5789aba0:::benchmark_data_smithi032_22800_object14673:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.759022+0000", 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0047186800000000003, 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048389660000000001, 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.771 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14674 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759022+0000", 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759022+0000", 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759022+0000", 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5499999999999997e-07 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759089+0000", 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6897999999999995e-05 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759099+0000", 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6490000000000003e-06 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759106+0000", 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.776e-06 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760108+0000", 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010021909999999999 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760257+0000", 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014859500000000001 2024-04-18T22:23:01.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760391+0000", 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013436599999999999 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762703+0000", 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002312223 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.775 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14675 5.5 5:a473fb0e:::benchmark_data_smithi032_22800_object14674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.759393+0000", 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.004347353, 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045183760000000002, 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14675 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759393+0000", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759393+0000", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759394+0000", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8200000000000008e-07 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759472+0000", 2024-04-18T22:23:01.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7881999999999997e-05 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759483+0000", 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1388e-05 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759491+0000", 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6179999999999997e-06 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759544+0000", 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2852999999999997e-05 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759661+0000", 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011748600000000001 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759790+0000", 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012907000000000001 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761478+0000", 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016876630000000001 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14676 5.a 5:5a15510a:::benchmark_data_smithi032_22800_object14675:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.759512+0000", 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0042286850000000003, 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044548820000000003, 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.780 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14676 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759512+0000", 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759512+0000", 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759512+0000", 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9999999999999997e-07 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759554+0000", 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1514999999999998e-05 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759562+0000", 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1459999999999996e-06 2024-04-18T22:23:01.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.759569+0000", 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4780000000000002e-06 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761005+0000", 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014355660000000001 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761089+0000", 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3652000000000005e-05 2024-04-18T22:23:01.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761165+0000", 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6402999999999993e-05 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762710+0000", 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015448009999999999 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14678 5.a 5:587c2161:::benchmark_data_smithi032_22800_object14677:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.760731+0000", 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030098019999999998, 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032803039999999999, 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14678 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760731+0000", 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760731+0000", 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760731+0000", 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3200000000000004e-07 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760849+0000", 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117827 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760859+0000", 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8660000000000001e-06 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760864+0000", 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761981+0000", 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011164599999999999 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762071+0000", 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0390999999999996e-05 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762150+0000", 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9262000000000001e-05 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.763382+0000", 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012314470000000001 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14679 5.c 5:310fd0d8:::benchmark_data_smithi032_22800_object14678:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.760874+0000", 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0028660030000000002, 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031892909999999999, 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14679 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760874+0000", 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760874+0000", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760875+0000", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760907+0000", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1856999999999997e-05 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760913+0000", 2024-04-18T22:23:01.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7050000000000002e-06 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760929+0000", 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5821e-05 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.760975+0000", 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6164e-05 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761103+0000", 2024-04-18T22:23:01.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012803200000000001 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761235+0000", 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144099999999999 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762652+0000", 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014168410000000001 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14680 5.7 5:ee568cd6:::benchmark_data_smithi032_22800_object14679:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.761119+0000", 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026217459999999999, 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030210269999999999, 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14680 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761119+0000", 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761119+0000", 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761119+0000", 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6799999999999999e-07 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761218+0000", 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8159999999999995e-05 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761227+0000", 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5629999999999998e-06 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761233+0000", 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4219999999999999e-06 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761284+0000", 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1727000000000001e-05 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761417+0000", 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013265099999999999 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.761548+0000", 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130975 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.763001+0000", 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001452685 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14681 5.3 5:ca50e9c5:::benchmark_data_smithi032_22800_object14680:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.762274+0000", 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014665940000000001, 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019092740000000001, 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14681 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762274+0000", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762274+0000", 2024-04-18T22:23:01.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762275+0000", 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3799999999999996e-07 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762318+0000", 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3157e-05 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762330+0000", 2024-04-18T22:23:01.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2106000000000001e-05 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762338+0000", 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0830000000000004e-06 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762381+0000", 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3173000000000002e-05 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762473+0000", 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1481e-05 2024-04-18T22:23:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.762598+0000", 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012499700000000001 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.763746+0000", 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011484010000000001 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:01.802 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 9 2024-04-18T22:23:01.803 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:01.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:01.804 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:01.804+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:01.904 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:01.902+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:01.965 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:01.965 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:01.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:01.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:23:02.033 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:02.035+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:02.106 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:02.102+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:02.149 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:02.149 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:02.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:02.149 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:02.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.149 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032941329, 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.155 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032808837000001, 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:02.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:02.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032737251, 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032608998000001, 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:02.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.173 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032499671, 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:21 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032329556000001, 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.027074913999999999, 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:23:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116210+0000", 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9190000000000002e-06 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116218+0000", 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.994e-06 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116239+0000", 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0491000000000001e-05 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116260+0000", 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1118e-05 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119672+0000", 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034115410000000001 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:02.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119685+0000", 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3866000000000001e-05 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.132805+0000", 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013119445 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133030+0000", 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022555700000000001 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133216+0000", 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018553999999999999 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140260+0000", 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070438760000000001 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.142993+0000", 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027334389999999998 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.143165+0000", 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000172165 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.143283+0000", 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117963 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032243334, 2024-04-18T22:23:02.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.189 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032155847999999, 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:02.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:02.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:02.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:02.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.032077862000001, 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:02.194 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.198 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(mds.0.3:37 3.13 e17/15 3:cd5a64a3:::100.00000000:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.022319278000001, 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014004429000000001, 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126220+0000", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1299999999999998e-07 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126247+0000", 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7328e-05 2024-04-18T22:23:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126267+0000", 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9655000000000001e-05 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126284+0000", 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7075000000000002e-05 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126429+0000", 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014528699999999999 2024-04-18T22:23:02.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126491+0000", 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1587999999999994e-05 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140139+0000", 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013648476 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140223+0000", 2024-04-18T22:23:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4107e-05 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:3 5.6 5:62098041:::benchmark_data_smithi032_22800_object2:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.9556837619999996, 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012453989, 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.203 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192857+0000", 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5780000000000001e-06 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193049+0000", 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019216900000000001 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193084+0000", 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5265999999999997e-05 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193098+0000", 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3295000000000001e-05 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193303+0000", 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020540599999999999 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193313+0000", 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0093e-05 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200364+0000", 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070508469999999998 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200467+0000", 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010301099999999999 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200628+0000", 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016122700000000001 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202430+0000", 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001801551 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205199+0000", 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027694490000000002 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205265+0000", 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5505000000000004e-05 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205308+0000", 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3591999999999999e-05 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:7 5.6 5:6a6275e8:::benchmark_data_smithi032_22800_object6:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.9553139929999999, 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013282308, 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:23:02.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193225+0000", 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.009e-06 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193457+0000", 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000232308 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193472+0000", 2024-04-18T22:23:02.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5101e-05 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193484+0000", 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1990999999999999e-05 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193553+0000", 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8149000000000001e-05 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193559+0000", 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5339999999999996e-06 2024-04-18T22:23:02.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201146+0000", 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075871929999999999 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201251+0000", 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104695 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201374+0000", 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012274200000000001 2024-04-18T22:23:02.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202530+0000", 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011561010000000001 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206367+0000", 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038374400000000001 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206448+0000", 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0486000000000002e-05 2024-04-18T22:23:02.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206506+0000", 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8559e-05 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:10 5.5 5:ac01b574:::benchmark_data_smithi032_22800_object9:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.9549063379999998, 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011974561999999999, 2024-04-18T22:23:02.215 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 10 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193632+0000", 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.37e-07 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193690+0000", 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7331999999999999e-05 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193697+0000", 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2139999999999999e-06 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193703+0000", 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.891e-06 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193760+0000", 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7358999999999997e-05 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193765+0000", 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8670000000000002e-06 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200394+0000", 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066286399999999999 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200540+0000", 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000146427 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200705+0000", 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000164871 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202046+0000", 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013406469999999999 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205489+0000", 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034432730000000002 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205555+0000", 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6476000000000005e-05 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205606+0000", 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0927999999999998e-05 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14 5.6 5:6abb6cd9:::benchmark_data_smithi032_22800_object13:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:02.221 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.9548237579999999, 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012915045999999999, 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193715+0000", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6699999999999999e-07 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193774+0000", 2024-04-18T22:23:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9286000000000001e-05 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193783+0000", 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6000000000000007e-06 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193788+0000", 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3639999999999998e-06 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193837+0000", 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8656000000000003e-05 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193841+0000", 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8489999999999999e-06 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201856+0000", 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0080156819999999997 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201955+0000", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8970999999999998e-05 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202087+0000", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013226199999999999 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202906+0000", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00081854200000000001 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206559+0000", 2024-04-18T22:23:02.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036528469999999999 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206600+0000", 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0794999999999998e-05 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206629+0000", 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9725000000000001e-05 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:12378 5.2 5:4f1ecb70:::benchmark_data_smithi032_22800_object12377:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:00.915762+0000", 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 1.232775613, 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012145032, 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 12378 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.228 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915762+0000", 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915762+0000", 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915763+0000", 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.02e-07 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915876+0000", 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112569 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915890+0000", 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4576000000000001e-05 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915901+0000", 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0314e-05 2024-04-18T22:23:02.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917894+0000", 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001992714 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917998+0000", 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104479 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918089+0000", 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1028999999999999e-05 2024-04-18T22:23:02.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.919420+0000", 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013310990000000001 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.927799+0000", 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083786299999999998 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.927859+0000", 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0222e-05 2024-04-18T22:23:02.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.927907+0000", 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8498000000000002e-05 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:12382 5.2 5:410264f5:::benchmark_data_smithi032_22800_object12381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:00.917953+0000", 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 1.230584755, 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012810045000000001, 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 12382 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917953+0000", 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917953+0000", 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917954+0000", 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1200000000000002e-07 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918042+0000", 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8121000000000006e-05 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918057+0000", 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4632e-05 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918065+0000", 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4470000000000006e-06 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918615+0000", 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054941600000000005 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918726+0000", 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111783 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918859+0000", 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013233099999999999 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.921093+0000", 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022346760000000001 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.930712+0000", 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0096183840000000007 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.930740+0000", 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7835e-05 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.930763+0000", 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3708000000000001e-05 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.238 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:13802 5.3 5:c37f646b:::benchmark_data_smithi032_22800_object13801:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.422715+0000", 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.72582307199999996, 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011991374000000001, 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 13802 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.239 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.422715+0000", 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.422715+0000", 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.422716+0000", 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6700000000000003e-07 2024-04-18T22:23:02.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.422755+0000", 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9397000000000002e-05 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.422764+0000", 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8789999999999992e-06 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.422770+0000", 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9190000000000004e-06 2024-04-18T22:23:02.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.424046+0000", 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001276332 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.424155+0000", 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010890700000000001 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.424251+0000", 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6028999999999998e-05 2024-04-18T22:23:02.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.425936+0000", 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001685001 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.434600+0000", 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0086636560000000005 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.434662+0000", 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1977999999999995e-05 2024-04-18T22:23:02.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.434706+0000", 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4608999999999999e-05 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14527 5.3 5:cae83e2a:::benchmark_data_smithi032_22800_object14526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.696410+0000", 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.452128156, 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016021106, 2024-04-18T22:23:02.244 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14527 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696410+0000", 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696410+0000", 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696411+0000", 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696484+0000", 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3648999999999999e-05 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696495+0000", 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0710000000000001e-05 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696506+0000", 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1055e-05 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696550+0000", 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4056e-05 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696661+0000", 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110314 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696808+0000", 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014779300000000001 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.702975+0000", 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0061665549999999998 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712331+0000", 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093556779999999992 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712386+0000", 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5668000000000003e-05 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712431+0000", 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4792000000000001e-05 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.249 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14540 5.3 5:cac2580d:::benchmark_data_smithi032_22800_object14539:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.44460851699999998, 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014209255, 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14540 2024-04-18T22:23:02.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:02.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7800000000000001e-07 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.704014+0000", 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3560999999999995e-05 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.704029+0000", 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4639e-05 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.704035+0000", 2024-04-18T22:23:02.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.708e-06 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.706003+0000", 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001967847 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.706168+0000", 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000165021 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.706252+0000", 2024-04-18T22:23:02.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3419999999999998e-05 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712478+0000", 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062267470000000004 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.717993+0000", 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055143910000000004 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.718024+0000", 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1084999999999998e-05 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.718139+0000", 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115058 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14942 5.3 5:c0ad4d45:::benchmark_data_smithi032_22800_object14941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.861831+0000", 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.286706929, 2024-04-18T22:23:02.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013316412999999999, 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14942 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861831+0000", 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861831+0000", 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861832+0000", 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3099999999999996e-07 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861905+0000", 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2554999999999999e-05 2024-04-18T22:23:02.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861915+0000", 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0488e-05 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861923+0000", 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.456e-06 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.863743+0000", 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001819415 2024-04-18T22:23:02.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.863883+0000", 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140254 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.864038+0000", 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000155239 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.865915+0000", 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018770340000000001 2024-04-18T22:23:02.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.875084+0000", 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091689779999999995 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.875119+0000", 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4643e-05 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.875148+0000", 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8520000000000001e-05 2024-04-18T22:23:02.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.261 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:02.261 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:02.261 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:02.262 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 6 16 15855 15839 164.973 169.375 0.0084492 0.00605114 2024-04-18T22:23:02.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:02.263 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:02.258+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:02.363 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:02.362+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:02.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:23:02.412 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:16451 5.d 5:baeebccb:::benchmark_data_smithi032_22800_object16450:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:02.408992+0000", 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0053921500000000001, 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054200749999999999, 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16451 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.408992+0000", 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.408992+0000", 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.408993+0000", 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999999e-07 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.409065+0000", 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1563999999999994e-05 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.409075+0000", 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0078000000000001e-05 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.409083+0000", 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.055e-06 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.409141+0000", 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999998e-05 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.409273+0000", 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013133900000000001 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.409413+0000", 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014077299999999999 2024-04-18T22:23:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410939+0000", 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001525782 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:16452 5.9 5:9afa7862:::benchmark_data_smithi032_22800_object16451:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:02.410299+0000", 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.004085137, 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041815339999999998, 2024-04-18T22:23:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16452 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410299+0000", 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410299+0000", 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410300+0000", 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410331+0000", 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0762000000000001e-05 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410342+0000", 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1382999999999999e-05 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410350+0000", 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2219999999999996e-06 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410393+0000", 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3479999999999997e-05 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410468+0000", 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4435000000000001e-05 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.410545+0000", 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7134999999999999e-05 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.412429+0000", 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018847 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:16463 5.8 5:1e24fb2f:::benchmark_data_smithi032_22800_object16462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:02.413500+0000", 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00088405700000000005, 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001027177, 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16463 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413500+0000", 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413500+0000", 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413501+0000", 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5700000000000002e-07 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413611+0000", 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109714 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413620+0000", 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1239999999999994e-06 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413625+0000", 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.826e-06 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413679+0000", 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4035e-05 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413837+0000", 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015813099999999999 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414016+0000", 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017883499999999999 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:16466 5.9 5:9447a6f6:::benchmark_data_smithi032_22800_object16465:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:02.413959+0000", 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00042546899999999999, 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061066199999999999, 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16466 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413959+0000", 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413959+0000", 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.413960+0000", 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000003e-07 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414103+0000", 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143552 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414112+0000", 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8869999999999998e-06 2024-04-18T22:23:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414116+0000", 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4179999999999999e-06 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414193+0000", 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6336999999999998e-05 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414290+0000", 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7532000000000001e-05 2024-04-18T22:23:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:02.414367+0000", 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6267999999999996e-05 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2024-04-18T22:23:02.430 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:02.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:02.475 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:02.474+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:02.528 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:02.548 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:02.548 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:02.564 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:02.563+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:02.569 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:02.622 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:02.623 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:02.645 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:02.665 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:02.664+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:02.766 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:02.767+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:02.798 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:02.798 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:02.817 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:02.867 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:02.866+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:02.968 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:02.967+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:02.970 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:02.970 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:02.989 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:03.068 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:03.067+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:03.129 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:03.130 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:03.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:03.170 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:03.166+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:03.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 7 16 18670 18654 166.537 175.938 0.00420823 0.00599726 2024-04-18T22:23:03.271 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:03.271+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:03.294 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:03.294 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:03.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:03.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:03.372+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:03.473 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:03.470+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:03.474 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:03.474 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:03.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:03.574 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:03.572+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:03.651 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:03.652 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:03.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:03.675 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:03.674+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:03.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:03.776+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:03.837 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:03.837 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:03.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:03.877 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:03.879+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:03.978 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:03.979+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:03.982 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:23:03.982 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:03.982 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-18T22:23:03.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-18T22:23:04.010 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:04.010 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:04.032 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:04.079 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:04.080+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:04.180 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:04.179+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:04.187 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:04.187 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:04.189 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 8 16 21358 21342 166.718 168 0.00657983 0.00599196 2024-04-18T22:23:04.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:04.281 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:04.283+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 25, 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "a1b9e9bc-8f1f-4426-9796-fae392070eec", 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-04-18T22:22:10.044535+0000", 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-04-18T22:23:02.392729+0000", 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-04-18T22:22:19.704344+0000", 2024-04-18T22:23:04.328 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-04-18T22:22:11.681436+0000", 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 4, 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-04-18T22:23:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 5, 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 3, 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "squid", 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-18T22:22:20.901399+0000", 2024-04-18T22:23:04.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-18T22:23:04.331 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-18T22:23:04.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "11", 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-18T22:23:04.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-18T22:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-18T22:23:04.335 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-18T22:23:04.336 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.66666668653488159, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.67000001668930054 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.337 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-18T22:22:26.016101+0000", 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-18T22:23:04.338 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-18T22:23:04.339 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "14", 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-18T22:23:04.340 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-18T22:23:04.341 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-18T22:23:04.342 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-18T22:23:04.343 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.25, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.25, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1.5, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.8799999952316284, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.8799999952316284, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.66666668653488159, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.67000001668930054 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-04-18T22:23:04.344 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-18T22:22:29.965363+0000", 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-18T22:23:04.345 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-18T22:23:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-18T22:23:04.347 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-18T22:23:04.348 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-18T22:23:04.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2024-04-18T22:23:04.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.190000057220459, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.190000057220459, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1.5, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.7799999713897705, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.7799999713897705, 2024-04-18T22:23:04.351 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.66666668653488159, 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.67000001668930054 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 4, 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-18T22:22:30.463150+0000", 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-18T22:23:04.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2024-04-18T22:23:04.353 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-18T22:23:04.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-18T22:23:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-18T22:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-18T22:23:04.357 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2024-04-18T22:23:04.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1, 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1, 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1.5, 2024-04-18T22:23:04.359 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.66666668653488159, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.67000001668930054 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 5, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "unique_pool_0", 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-04-18T22:22:53.788053+0000", 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-04-18T22:23:04.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 16, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 16, 2024-04-18T22:23:04.361 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 16, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 16, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 16, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "21", 2024-04-18T22:23:04.362 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-04-18T22:23:04.363 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-04-18T22:23:04.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-04-18T22:23:04.365 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "rados": {} 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "score_type": "Fair distribution", 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1.5, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.5, 2024-04-18T22:23:04.366 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.5, 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.66666668653488159, 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.67000001668930054 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "57238913-8d4f-4ea8-a1a5-92ce7aceaf4a", 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-18T22:23:04.367 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 8, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6800", 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6801", 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.369 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6802", 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6803", 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6806", 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.371 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6807", 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6804", 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.86:6805", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4267325794 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.86:6801/4267325794", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.86:6803/4267325794", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.86:6807/4267325794", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.86:6805/4267325794", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-18T22:23:04.373 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "68bfe465-447e-4083-a8d5-e6f996b2bb2c", 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-18T22:23:04.374 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 8, 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 23, 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6800", 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6801", 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6802", 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6803", 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6806", 2024-04-18T22:23:04.377 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6807", 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6804", 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.96:6805", 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3490435725 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.379 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.96:6801/3490435725", 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.96:6803/3490435725", 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3490435725", 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3490435725", 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e", 2024-04-18T22:23:04.380 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 8, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 19, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6800", 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6801", 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6802", 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6803", 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6806", 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6807", 2024-04-18T22:23:04.384 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6804", 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.196:6805", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2337964302 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.196:6801/2337964302", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.196:6803/2337964302", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.196:6807/2337964302", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.196:6805/2337964302", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-04-18T22:23:04.386 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-18T22:22:16.941980+0000", 2024-04-18T22:23:04.387 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-18T22:22:16.992073+0000", 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540701547738038271, 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-04-18T22:22:16.978727+0000", 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-04-18T22:23:04.389 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-04-18T22:23:04.390 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-04-18T22:23:04.391 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-04-18T22:23:04.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-18T22:23:04.392 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:23:04.393 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:04.395+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:04.394 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:04.394 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:04.395 INFO:tasks.thrashosds.thrasher:j is {'epoch': 25, 'fsid': 'a1b9e9bc-8f1f-4426-9796-fae392070eec', 'created': '2024-04-18T22:22:10.044535+0000', 'modified': '2024-04-18T22:23:02.392729+0000', 'last_up_change': '2024-04-18T22:22:19.704344+0000', 'last_in_change': '2024-04-18T22:22:11.681436+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 5, 'max_osd': 3, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-18T22:22:20.901399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6666666865348816, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.6700000166893005}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-18T22:22:26.016101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1.5, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.6666666865348816, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.6700000166893005}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-18T22:22:29.965363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.190000057220459, 'score_stable': 1.190000057220459, 'optimal_score': 1.5, 'raw_score_acting': 1.7799999713897705, 'raw_score_stable': 1.7799999713897705, 'primary_affinity_weighted': 0.6666666865348816, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.6700000166893005}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-04-18T22:22:30.463150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1, 'score_stable': 1, 'optimal_score': 1.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.6666666865348816, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.6700000166893005}}, {'pool': 5, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-18T22:22:53.788053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1, 'score_stable': 1, 'optimal_score': 1.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.6666666865348816, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.6700000166893005}}], 'osds': [{'osd': 0, 'uuid': '57238913-8d4f-4ea8-a1a5-92ce7aceaf4a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6800', 'nonce': 4267325794}, {'type': 'v1', 'addr': '172.21.15.86:6801', 'nonce': 4267325794}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6802', 'nonce': 4267325794}, {'type': 'v1', 'addr': '172.21.15.86:6803', 'nonce': 4267325794}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6806', 'nonce': 4267325794}, {'type': 'v1', 'addr': '172.21.15.86:6807', 'nonce': 4267325794}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6804', 'nonce': 4267325794}, {'type': 'v1', 'addr': '172.21.15.86:6805', 'nonce': 4267325794}]}, 'public_addr': '172.21.15.86:6801/4267325794', 'cluster_addr': '172.21.15.86:6803/4267325794', 'heartbeat_back_addr': '172.21.15.86:6807/4267325794', 'heartbeat_front_addr': '172.21.15.86:6805/4267325794', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '68bfe465-447e-4083-a8d5-e6f996b2bb2c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3490435725}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3490435725}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3490435725}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3490435725}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3490435725}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3490435725}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3490435725}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3490435725}]}, 'public_addr': '172.21.15.96:6801/3490435725', 'cluster_addr': '172.21.15.96:6803/3490435725', 'heartbeat_back_addr': '172.21.15.96:6807/3490435725', 'heartbeat_front_addr': '172.21.15.96:6805/3490435725', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6800', 'nonce': 2337964302}, {'type': 'v1', 'addr': '172.21.15.196:6801', 'nonce': 2337964302}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6802', 'nonce': 2337964302}, {'type': 'v1', 'addr': '172.21.15.196:6803', 'nonce': 2337964302}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6806', 'nonce': 2337964302}, {'type': 'v1', 'addr': '172.21.15.196:6807', 'nonce': 2337964302}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6804', 'nonce': 2337964302}, {'type': 'v1', 'addr': '172.21.15.196:6805', 'nonce': 2337964302}]}, 'public_addr': '172.21.15.196:6801/2337964302', 'cluster_addr': '172.21.15.196:6803/2337964302', 'heartbeat_back_addr': '172.21.15.196:6807/2337964302', 'heartbeat_front_addr': '172.21.15.196:6805/2337964302', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-18T22:22:16.941980+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-18T22:22:16.992073+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-18T22:22:16.978727+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-18T22:23:04.395 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-04-18T22:23:04.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:04.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:23:04.494 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:04.492+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:04.545 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:04.545 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:04.565 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:04.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:04.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:04.612+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:04.695 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:04.694+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:04.739 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:04.739 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:04.759 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:04.763 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:04.763 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:04.778 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:04.784 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:04.830 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:04.832+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:04.897 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:04.894+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:04.928 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:04.928 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4144.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e10)", 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.466352980000003, 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025591307000000001, 2024-04-18T22:23:04.931 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4144", 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1049496302", 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 141 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461621+0000", 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461622+0000", 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9400000000000004e-07 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461627+0000", 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6559999999999997e-06 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461638+0000", 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0594000000000001e-05 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.461645+0000", 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2099999999999996e-06 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480273+0000", 2024-04-18T22:23:04.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.018627812000000001 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480507+0000", 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00023432800000000001 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.480828+0000", 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032085799999999998 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.482445+0000", 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001617081 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487064+0000", 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046189179999999996 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487149+0000", 2024-04-18T22:23:04.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4575999999999998e-05 2024-04-18T22:23:04.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:22.487212+0000", 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3479999999999995e-05 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:6 3.10 3:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.938 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.813017158000001, 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027601216000000001, 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114957+0000", 2024-04-18T22:23:04.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114960+0000", 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.88e-06 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.114981+0000", 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0877999999999999e-05 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115008+0000", 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7617999999999999e-05 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115031+0000", 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2302000000000001e-05 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121370+0000", 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0063390540000000002 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121632+0000", 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000262539 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121921+0000", 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028841099999999997 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135317+0000", 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013396105 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142295+0000", 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0069786539999999999 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142445+0000", 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000149968 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142558+0000", 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011280700000000001 2024-04-18T22:23:04.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812904596999999, 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.944 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:04.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.949 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812835999000001, 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:04.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:04.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:04.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:04.956 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812740153, 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:04.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:04.962 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812683270999997, 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.963 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:04.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.969 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812623598999998, 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:04.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:04.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812565636999999, 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:04.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:15 3.14 3:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812501019000003, 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027152815, 2024-04-18T22:23:04.980 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115473+0000", 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115474+0000", 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8700000000000004e-07 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115482+0000", 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2560000000000002e-06 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115500+0000", 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8224999999999999e-05 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115514+0000", 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3648999999999999e-05 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115656+0000", 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000141865 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115959+0000", 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030357600000000001 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116299+0000", 2024-04-18T22:23:04.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033980600000000001 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126282+0000", 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0099824879999999994 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142386+0000", 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016104535999999999 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142517+0000", 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013121000000000001 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142626+0000", 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108317 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:16 3.1c 3:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812409483000003, 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.027983006000000001, 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115565+0000", 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115566+0000", 2024-04-18T22:23:04.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.127e-06 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115573+0000", 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0559999999999999e-06 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115591+0000", 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8615e-05 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115608+0000", 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6815000000000001e-05 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117673+0000", 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020651689999999999 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.118082+0000", 2024-04-18T22:23:04.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040828099999999998 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.118459+0000", 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037744699999999999 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126370+0000", 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079108990000000007 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143152+0000", 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016782488000000002 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143434+0000", 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000281988 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143548+0000", 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000113121 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812345944999997, 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:04.993 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:04.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:04.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:04.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:04.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:04.999 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812289016000001, 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:05.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:05.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812193129999997, 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:05.005 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:05.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:05.010 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812130801999999, 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:05.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:05.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.812058323999999, 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:05.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:05.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:2 3.f 3:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.811998092000003, 2024-04-18T22:23:05.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026491475, 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115976+0000", 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.06e-06 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115983+0000", 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2820000000000002e-06 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116005+0000", 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1525000000000001e-05 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116018+0000", 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3372e-05 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116290+0000", 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000271453 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116676+0000", 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038682899999999999 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117663+0000", 2024-04-18T22:23:05.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00098642700000000001 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125645+0000", 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079826340000000006 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142118+0000", 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.016472647 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142261+0000", 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014285799999999999 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.142467+0000", 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000206388 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:05.031 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.811905856000003, 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:05.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.811827133000001, 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:05.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:05.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.810306277000002, 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:05.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:05.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.048 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.810207925, 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:05.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:05.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:05.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:05.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:05.053 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.054 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.054 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.054 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:05.054 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:05.054 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:05.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:05.057 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:05.055+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:05.156 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:05.156+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:05.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 9 16 24107 24091 167.282 171.812 0.00542325 0.00597182 2024-04-18T22:23:05.229 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:05.229 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:05.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.229 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24192 5.5 5:afcdbb7a:::benchmark_data_smithi032_22800_object24191:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.229 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.226769+0000", 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0035927929999999999, 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036248309999999998, 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24192 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226769+0000", 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226769+0000", 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226769+0000", 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5700000000000002e-07 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226810+0000", 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0815000000000001e-05 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226821+0000", 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0550000000000001e-05 2024-04-18T22:23:05.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226828+0000", 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3599999999999998e-06 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227037+0000", 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020881799999999999 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227171+0000", 2024-04-18T22:23:05.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013407800000000001 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227302+0000", 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130734 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230187+0000", 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028849779999999998 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.234 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24194 5.a 5:5bd963fd:::benchmark_data_smithi032_22800_object24193:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.226929+0000", 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0034327559999999999, 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035921880000000001, 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24194 2024-04-18T22:23:05.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226929+0000", 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226929+0000", 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226929+0000", 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226965+0000", 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6000000000000001e-05 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226972+0000", 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.878e-06 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226980+0000", 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6350000000000006e-06 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227172+0000", 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000192475 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227294+0000", 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012149199999999999 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227420+0000", 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012650000000000001 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229872+0000", 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024519199999999998 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.239 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24196 5.6 5:670923e3:::benchmark_data_smithi032_22800_object24195:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.226996+0000", 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0033648129999999999, 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003669613, 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24196 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226996+0000", 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226996+0000", 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.226997+0000", 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1399999999999997e-07 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227031+0000", 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3762e-05 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227038+0000", 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4590000000000001e-06 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227044+0000", 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7899999999999996e-06 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227180+0000", 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000136284 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227317+0000", 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013629500000000001 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.227455+0000", 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013791399999999999 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229289+0000", 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001834671 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24201 5.7 5:e67d1ca0:::benchmark_data_smithi032_22800_object24200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.228247+0000", 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002114449, 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002514545, 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.244 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24201 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228247+0000", 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228247+0000", 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228248+0000", 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3900000000000004e-07 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228370+0000", 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012270200000000001 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228386+0000", 2024-04-18T22:23:05.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5580999999999999e-05 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228395+0000", 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4150000000000005e-06 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228456+0000", 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0177000000000003e-05 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228547+0000", 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1092000000000001e-05 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228710+0000", 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000163411 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24202 5.5 5:a4ac7b48:::benchmark_data_smithi032_22800_object24201:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.228644+0000", 2024-04-18T22:23:05.248 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0017174480000000001, 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022020009999999999, 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24202 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228644+0000", 2024-04-18T22:23:05.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228644+0000", 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228645+0000", 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7699999999999992e-07 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228757+0000", 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112395 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228767+0000", 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7030000000000001e-06 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228775+0000", 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6329999999999996e-06 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228848+0000", 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2933000000000001e-05 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229018+0000", 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017071400000000001 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229162+0000", 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014337400000000001 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:24198 5.e e25/19 5:789fc953:::benchmark_data_smithi032_22800_object24197:head v 25'1494, mlcod=25'1491)", 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.228721+0000", 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001640669, 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022023059999999998, 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228721+0000", 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228721+0000", 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228721+0000", 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9699999999999996e-07 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228898+0000", 2024-04-18T22:23:05.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177314 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228922+0000", 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3312999999999999e-05 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228930+0000", 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8730000000000007e-06 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228982+0000", 2024-04-18T22:23:05.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2537000000000002e-05 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229003+0000", 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0849999999999999e-05 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24203 5.2 5:4526302d:::benchmark_data_smithi032_22800_object24202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.228863+0000", 2024-04-18T22:23:05.256 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001498442, 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021297999999999998, 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24203 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228863+0000", 2024-04-18T22:23:05.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228863+0000", 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228864+0000", 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1500000000000003e-07 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228939+0000", 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5261999999999999e-05 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228950+0000", 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0896e-05 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228958+0000", 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6599999999999995e-06 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229200+0000", 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00024268599999999999 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229371+0000", 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000171034 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229510+0000", 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013867400000000001 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:24200 5.4 e25/19 5:2dc823e3:::benchmark_data_smithi032_22800_object24199:head v 25'1543, mlcod=25'1539)", 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.228951+0000", 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001410763, 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021426790000000002, 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228951+0000", 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228951+0000", 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228951+0000", 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1200000000000003e-07 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228991+0000", 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0080999999999998e-05 2024-04-18T22:23:05.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.228999+0000", 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2600000000000005e-06 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229005+0000", 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6099999999999997e-06 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229051+0000", 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5606999999999999e-05 2024-04-18T22:23:05.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229070+0000", 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9431000000000001e-05 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:24199 5.e e25/19 5:7dcac854:::benchmark_data_smithi032_22800_object24198:head v 25'1495, mlcod=25'1491)", 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.229827+0000", 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00053389500000000005, 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013298170000000001, 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229827+0000", 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229827+0000", 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229828+0000", 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.68e-07 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229883+0000", 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5340999999999998e-05 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229893+0000", 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6330000000000008e-06 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229900+0000", 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9249999999999998e-06 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230144+0000", 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00024400800000000001 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230161+0000", 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6752000000000002e-05 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.267 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:24204 5.5 5:a1ac3e1d:::benchmark_data_smithi032_22800_object24203:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.229912+0000", 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00044950900000000001, 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001303837, 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 24204 2024-04-18T22:23:05.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229912+0000", 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229912+0000", 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229913+0000", 2024-04-18T22:23:05.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.251e-06 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229948+0000", 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5327999999999998e-05 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229959+0000", 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0345e-05 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.229966+0000", 2024-04-18T22:23:05.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230234+0000", 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267476 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230314+0000", 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9832000000000004e-05 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230391+0000", 2024-04-18T22:23:05.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7768999999999994e-05 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:24205 5.4 e25/19 5:2d5fa4f0:::benchmark_data_smithi032_22800_object24204:head v 25'1544, mlcod=25'1540)", 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.230029+0000", 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00033259700000000001, 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012584720000000001, 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230029+0000", 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230029+0000", 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230029+0000", 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3399999999999999e-07 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230098+0000", 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8763999999999994e-05 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230109+0000", 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1182999999999999e-05 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230116+0000", 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6980000000000001e-06 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230251+0000", 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013516600000000001 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.230262+0000", 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1168e-05 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 11 2024-04-18T22:23:05.275 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:05.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:05.277 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:05.275+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:05.378 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:05.374+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:05.477 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:05.478 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:05.474+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:05.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:05.580 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:05.580+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:05.660 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:05.660 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:05.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:05.660 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:05.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.660 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544847654999998, 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:05.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544715162999999, 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:05.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:05.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544643577000002, 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.673 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:05.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:05.679 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544515324000002, 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:05.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.685 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544405996999998, 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:05.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.691 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:21 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544235882000002, 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.027074913999999999, 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116208+0000", 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116210+0000", 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9190000000000002e-06 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116218+0000", 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.994e-06 2024-04-18T22:23:05.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116239+0000", 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0491000000000001e-05 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116260+0000", 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1118e-05 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119672+0000", 2024-04-18T22:23:05.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034115410000000001 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119685+0000", 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3866000000000001e-05 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.132805+0000", 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013119445 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133030+0000", 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022555700000000001 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133216+0000", 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018553999999999999 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140260+0000", 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070438760000000001 2024-04-18T22:23:05.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.142993+0000", 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027334389999999998 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.143165+0000", 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000172165 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.143283+0000", 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117963 2024-04-18T22:23:05.697 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544149660000002, 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.698 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:05.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:05.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:05.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.544062173999997, 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:05.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:05.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.543984188000003, 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:05.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(mds.0.3:37 3.13 e17/15 3:cd5a64a3:::100.00000000:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 33.534225604, 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014004429000000001, 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126219+0000", 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126220+0000", 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1299999999999998e-07 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126247+0000", 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7328e-05 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126267+0000", 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9655000000000001e-05 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126284+0000", 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7075000000000002e-05 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126429+0000", 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014528699999999999 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.126491+0000", 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1587999999999994e-05 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140139+0000", 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013648476 2024-04-18T22:23:05.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140223+0000", 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4107e-05 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:3 5.6 5:62098041:::benchmark_data_smithi032_22800_object2:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.4675900879999997, 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012453989, 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 3 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192854+0000", 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.192857+0000", 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5780000000000001e-06 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193049+0000", 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019216900000000001 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193084+0000", 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5265999999999997e-05 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193098+0000", 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3295000000000001e-05 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193303+0000", 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020540599999999999 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193313+0000", 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0093e-05 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200364+0000", 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0070508469999999998 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200467+0000", 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010301099999999999 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.200628+0000", 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016122700000000001 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202430+0000", 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001801551 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205199+0000", 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027694490000000002 2024-04-18T22:23:05.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205265+0000", 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5505000000000004e-05 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.205308+0000", 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3591999999999999e-05 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.723 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:7 5.6 5:6a6275e8:::benchmark_data_smithi032_22800_object6:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.4672203190000008, 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013282308, 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 7 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193224+0000", 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193225+0000", 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.009e-06 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193457+0000", 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000232308 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193472+0000", 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5101e-05 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193484+0000", 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1990999999999999e-05 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193553+0000", 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8149000000000001e-05 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193559+0000", 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5339999999999996e-06 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201146+0000", 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075871929999999999 2024-04-18T22:23:05.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201251+0000", 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104695 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201374+0000", 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012274200000000001 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202530+0000", 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011561010000000001 2024-04-18T22:23:05.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206367+0000", 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038374400000000001 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206448+0000", 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0486000000000002e-05 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206506+0000", 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8559e-05 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14 5.6 5:6abb6cd9:::benchmark_data_smithi032_22800_object13:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.4667300839999999, 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012915045999999999, 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193714+0000", 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193715+0000", 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6699999999999999e-07 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193774+0000", 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9286000000000001e-05 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193783+0000", 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6000000000000007e-06 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193788+0000", 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3639999999999998e-06 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193837+0000", 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8656000000000003e-05 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:05.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.193841+0000", 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8489999999999999e-06 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201856+0000", 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0080156819999999997 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.201955+0000", 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8970999999999998e-05 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202087+0000", 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013226199999999999 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.202906+0000", 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00081854200000000001 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206559+0000", 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036528469999999999 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206600+0000", 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0794999999999998e-05 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:56.206629+0000", 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9725000000000001e-05 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.736 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:12378 5.2 5:4f1ecb70:::benchmark_data_smithi032_22800_object12377:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:00.915762+0000", 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.7446819390000003, 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012145032, 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 12378 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915762+0000", 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915762+0000", 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915763+0000", 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.02e-07 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915876+0000", 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112569 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915890+0000", 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4576000000000001e-05 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.915901+0000", 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0314e-05 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917894+0000", 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001992714 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917998+0000", 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104479 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918089+0000", 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1028999999999999e-05 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.919420+0000", 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013310990000000001 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.927799+0000", 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083786299999999998 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.927859+0000", 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0222e-05 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.927907+0000", 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8498000000000002e-05 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:12382 5.2 5:410264f5:::benchmark_data_smithi032_22800_object12381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:00.917953+0000", 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.7424910809999998, 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012810045000000001, 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 12382 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917953+0000", 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917953+0000", 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.917954+0000", 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1200000000000002e-07 2024-04-18T22:23:05.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918042+0000", 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8121000000000006e-05 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918057+0000", 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4632e-05 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918065+0000", 2024-04-18T22:23:05.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4470000000000006e-06 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918615+0000", 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054941600000000005 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918726+0000", 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111783 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.918859+0000", 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013233099999999999 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.921093+0000", 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022346760000000001 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.930712+0000", 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0096183840000000007 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.930740+0000", 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7835e-05 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:00.930763+0000", 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3708000000000001e-05 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.748 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14527 5.3 5:cae83e2a:::benchmark_data_smithi032_22800_object14526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.696410+0000", 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.9640344819999997, 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016021106, 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14527 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696410+0000", 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696410+0000", 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696411+0000", 2024-04-18T22:23:05.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696484+0000", 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3648999999999999e-05 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696495+0000", 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0710000000000001e-05 2024-04-18T22:23:05.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696506+0000", 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1055e-05 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696550+0000", 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4056e-05 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696661+0000", 2024-04-18T22:23:05.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110314 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.696808+0000", 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014779300000000001 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.702975+0000", 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0061665549999999998 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712331+0000", 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093556779999999992 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712386+0000", 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5668000000000003e-05 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712431+0000", 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4792000000000001e-05 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.754 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14540 5.3 5:cac2580d:::benchmark_data_smithi032_22800_object14539:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.9565148429999999, 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014209255, 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14540 2024-04-18T22:23:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.703930+0000", 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7800000000000001e-07 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.704014+0000", 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3560999999999995e-05 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.704029+0000", 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4639e-05 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.704035+0000", 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.708e-06 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.706003+0000", 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001967847 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.706168+0000", 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000165021 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.706252+0000", 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3419999999999998e-05 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.712478+0000", 2024-04-18T22:23:05.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062267470000000004 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.717993+0000", 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055143910000000004 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.718024+0000", 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1084999999999998e-05 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.718139+0000", 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115058 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:14942 5.3 5:c0ad4d45:::benchmark_data_smithi032_22800_object14941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:01.861831+0000", 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.7986132550000002, 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013316412999999999, 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.761 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 14942 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861831+0000", 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861831+0000", 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861832+0000", 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3099999999999996e-07 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861905+0000", 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2554999999999999e-05 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861915+0000", 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0488e-05 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.861923+0000", 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.456e-06 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.863743+0000", 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001819415 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.863883+0000", 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140254 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.864038+0000", 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000155239 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.865915+0000", 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018770340000000001 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.875084+0000", 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091689779999999995 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.875119+0000", 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4643e-05 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:01.875148+0000", 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8520000000000001e-05 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25170 5.c 5:3f80da74:::benchmark_data_smithi032_22800_object25169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.601673+0000", 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.058771858000000003, 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013235485, 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25170 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.601673+0000", 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.601673+0000", 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.601673+0000", 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999996e-07 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.601761+0000", 2024-04-18T22:23:05.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7730000000000002e-05 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.601774+0000", 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2448e-05 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.601782+0000", 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3019999999999995e-06 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604641+0000", 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028589100000000001 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604778+0000", 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013692500000000001 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604929+0000", 2024-04-18T22:23:05.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015137899999999999 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.607133+0000", 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002204148 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.614818+0000", 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0076847199999999999 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.614871+0000", 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3396000000000002e-05 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.614908+0000", 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6689000000000001e-05 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25177 5.3 5:c71063c7:::benchmark_data_smithi032_22800_object25176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.604849+0000", 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.055595660999999998, 2024-04-18T22:23:05.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014237545000000001, 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25177 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604849+0000", 2024-04-18T22:23:05.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604849+0000", 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604850+0000", 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9300000000000006e-07 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:05.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604955+0000", 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105064 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604963+0000", 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1550000000000007e-06 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.604968+0000", 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.147e-06 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.605006+0000", 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8173999999999997e-05 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.605085+0000", 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8573999999999997e-05 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.605186+0000", 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100849 2024-04-18T22:23:05.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.606563+0000", 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013772260000000001 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.618972+0000", 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012409125999999999 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.619046+0000", 2024-04-18T22:23:05.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3851999999999997e-05 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.619086+0000", 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0385e-05 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:05.779 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:05.781 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:05.781 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:05.780+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:05.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:05.880+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25702 5.e 5:71461507:::benchmark_data_smithi032_22800_object25701:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.909075+0000", 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0089515489999999996, 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0089727360000000003, 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25702 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909075+0000", 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909075+0000", 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909077+0000", 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.057e-06 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909133+0000", 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5889e-05 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909162+0000", 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9516e-05 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909177+0000", 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4476e-05 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909287+0000", 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110435 2024-04-18T22:23:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909553+0000", 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026552200000000002 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.909782+0000", 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022963899999999999 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.912063+0000", 2024-04-18T22:23:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022806770000000001 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25707 5.b 5:df1e061d:::benchmark_data_smithi032_22800_object25706:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.911540+0000", 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0064860960000000002, 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0065626800000000004, 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25707 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911540+0000", 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911540+0000", 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911542+0000", 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.522e-06 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911593+0000", 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1180999999999997e-05 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911609+0000", 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6075000000000001e-05 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911619+0000", 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0248000000000001e-05 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911684+0000", 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4394999999999996e-05 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911826+0000", 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014264099999999999 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.911960+0000", 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013371300000000001 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915834+0000", 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003873881 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.926 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25708 5.f 5:f023daf2:::benchmark_data_smithi032_22800_object25707:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.913400+0000", 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0046264339999999996, 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047491149999999999, 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25708 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913400+0000", 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913400+0000", 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913402+0000", 2024-04-18T22:23:05.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913486+0000", 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4135000000000007e-05 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913518+0000", 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1835999999999998e-05 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913559+0000", 2024-04-18T22:23:05.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0386000000000003e-05 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913636+0000", 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7432e-05 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913765+0000", 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129033 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913993+0000", 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022773300000000001 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.917181+0000", 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031885289999999998 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25709 5.8 5:15bbb0f6:::benchmark_data_smithi032_22800_object25708:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.913631+0000", 2024-04-18T22:23:05.931 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0043958390000000003, 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045642740000000001, 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25709 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913631+0000", 2024-04-18T22:23:05.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913631+0000", 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913632+0000", 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.53e-06 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913716+0000", 2024-04-18T22:23:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3574999999999999e-05 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913738+0000", 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1963e-05 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913754+0000", 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6226999999999999e-05 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913871+0000", 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011697999999999999 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914113+0000", 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000242184 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914385+0000", 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027193699999999998 2024-04-18T22:23:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25710 5.e 5:79ea7c9f:::benchmark_data_smithi032_22800_object25709:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.913784+0000", 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0042422789999999998, 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044517610000000003, 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25710 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913784+0000", 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913784+0000", 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913785+0000", 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3340000000000001e-06 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913859+0000", 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3034999999999994e-05 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913879+0000", 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0302999999999999e-05 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913893+0000", 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4535999999999999e-05 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.913960+0000", 2024-04-18T22:23:05.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6353999999999995e-05 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914079+0000", 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118814 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914220+0000", 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141211 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.916489+0000", 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022696029999999998 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25712 5.4 5:2bf835b8:::benchmark_data_smithi032_22800_object25711:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.914189+0000", 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0038370729999999999, 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004090537, 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25712 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914189+0000", 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914189+0000", 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914191+0000", 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6199999999999999e-06 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914257+0000", 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6371999999999994e-05 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914280+0000", 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2843000000000001e-05 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914298+0000", 2024-04-18T22:23:05.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8162999999999999e-05 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914436+0000", 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013763299999999999 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914708+0000", 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027204300000000002 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914966+0000", 2024-04-18T22:23:05.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025833900000000001 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25713 5.8 5:182b0a4e:::benchmark_data_smithi032_22800_object25712:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.914375+0000", 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0036511859999999998, 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039478300000000003, 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25713 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914375+0000", 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914375+0000", 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914377+0000", 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-06 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914432+0000", 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5198999999999998e-05 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914452+0000", 2024-04-18T22:23:05.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9720000000000001e-05 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914467+0000", 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5342999999999999e-05 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.916556+0000", 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020893449999999998 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.916726+0000", 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016956999999999999 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.916900+0000", 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000174284 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25715 5.0 5:0d3e1699:::benchmark_data_smithi032_22800_object25714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.914827+0000", 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0031992589999999999, 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035379690000000002, 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25715 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914827+0000", 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914827+0000", 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914829+0000", 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.457e-06 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914892+0000", 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3388999999999999e-05 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914914+0000", 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2342999999999999e-05 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.914930+0000", 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5605999999999999e-05 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.916551+0000", 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016212939999999999 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.916814+0000", 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026236199999999998 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.917109+0000", 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029524 2024-04-18T22:23:05.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4586.0:25707 5.b e26/19)", 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.918368+0000", 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9996581, 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05, 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:05.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.918368+0000", 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.918368+0000", 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.918369+0000", 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.06e-06 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:05.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.918376+0000", 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.918392+0000", 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6203000000000001e-05 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.918402+0000", 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9399999999999997e-06 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:05.958 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:05.958 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 9 2024-04-18T22:23:05.958 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:05.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:05.982 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:05.984+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:06.083 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:06.083+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:06.112 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:06.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:06.184 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:06.182+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:06.190 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 10 16 26345 26329 164.54 139.875 0.00838852 0.00607166 2024-04-18T22:23:06.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:06.284+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.178122979000001, 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:23:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175789086999998, 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:23:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:23:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:23:06.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:23:06.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175686188, 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:23:06.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175636519999998, 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:23:06.310 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:23:06.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:23:06.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:23:06.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175570845000003, 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:06.321 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175511098000001, 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:23:06.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:23:06.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175431027999998, 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.327 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:23:06.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:23:06.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:23:06.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175369619999998, 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:23:06.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.338 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.175285307000003, 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:23:06.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:23:06.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.168129573999998, 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:23:06.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:23:06.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:23:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25664 5.f 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48962720799999998, 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093405780999999993, 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25664 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802645+0000", 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013290900000000001 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802656+0000", 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-05 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802661+0000", 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6729999999999997e-06 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802711+0000", 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9329000000000002e-05 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802804+0000", 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2929000000000004e-05 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802894+0000", 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0346999999999994e-05 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.807400+0000", 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045058290000000003 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895676+0000", 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088275705999999995 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895810+0000", 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134635 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895917+0000", 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107309 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25666 5.f 5:f7add380:::benchmark_data_smithi032_22800_object25665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48945369100000002, 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093393754999999995, 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25666 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802686+0000", 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802716+0000", 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9954000000000002e-05 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802723+0000", 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5689999999999999e-06 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802728+0000", 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6079999999999998e-06 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803716+0000", 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098778600000000005 2024-04-18T22:23:06.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803871+0000", 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015491100000000001 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.804071+0000", 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020055600000000001 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.810020+0000", 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059493339999999997 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895982+0000", 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085962083999999994 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896036+0000", 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3813999999999997e-05 2024-04-18T22:23:06.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896079+0000", 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2676000000000003e-05 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25673 5.d 5:becd3ab4:::benchmark_data_smithi032_22800_object25672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48408617500000001, 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085178675999999995, 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25673 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808054+0000", 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2900000000000002e-07 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808114+0000", 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9993e-05 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808126+0000", 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808136+0000", 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.064e-05 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808186+0000", 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9471000000000001e-05 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808277+0000", 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0906e-05 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808387+0000", 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011026099999999999 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.812706+0000", 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043185289999999998 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.892987+0000", 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080281292000000004 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893126+0000", 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013872099999999999 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893231+0000", 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010582300000000001 2024-04-18T22:23:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25675 5.f 5:f60cbd32:::benchmark_data_smithi032_22800_object25674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.47664460400000003, 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082348202999999995, 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25675 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815495+0000", 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815546+0000", 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0933999999999997e-05 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815572+0000", 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5936999999999999e-05 2024-04-18T22:23:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815585+0000", 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2860000000000001e-05 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815664+0000", 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9334999999999999e-05 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815838+0000", 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017410700000000001 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.816021+0000", 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018247599999999999 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.817763+0000", 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001742494 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897556+0000", 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079792478999999999 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897717+0000", 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161199 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897843+0000", 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125415 2024-04-18T22:23:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25682 5.b 5:d19b5078:::benchmark_data_smithi032_22800_object25681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.398327243, 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017001512999999999, 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25682 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4899999999999995e-07 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893846+0000", 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3546e-05 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893855+0000", 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5689999999999994e-06 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893862+0000", 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7070000000000003e-06 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898400+0000", 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045379089999999997 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898589+0000", 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188965 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898831+0000", 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000242181 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902115+0000", 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032839369999999998 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910719+0000", 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086036350000000001 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910771+0000", 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2584999999999999e-05 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910813+0000", 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1829999999999998e-05 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25693 5.e 5:715faa37:::benchmark_data_smithi032_22800_object25692:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39291904500000002, 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016385088999999999, 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25693 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899222+0000", 2024-04-18T22:23:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.646e-06 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899274+0000", 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2642999999999997e-05 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899292+0000", 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8196e-05 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899304+0000", 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1866e-05 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899425+0000", 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121131 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899751+0000", 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032516000000000001 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.900183+0000", 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043271999999999999 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902912+0000", 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027285930000000001 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915375+0000", 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012463093 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915503+0000", 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012793099999999999 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915605+0000", 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010211 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26085 5.f 5:ffd60b63:::benchmark_data_smithi032_22800_object26084:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.086875+0000", 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.20526348999999999, 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015390157, 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26085 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086875+0000", 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086875+0000", 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086876+0000", 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2900000000000003e-07 2024-04-18T22:23:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086922+0000", 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6449000000000002e-05 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086934+0000", 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1911e-05 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086939+0000", 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4730000000000002e-06 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086998+0000", 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9401999999999998e-05 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087212+0000", 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021393700000000001 2024-04-18T22:23:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087470+0000", 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025744800000000002 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092867+0000", 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053969029999999998 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102165+0000", 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092985849999999998 2024-04-18T22:23:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102212+0000", 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6807000000000001e-05 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102266+0000", 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3613000000000003e-05 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26086 5.f 5:fca465c3:::benchmark_data_smithi032_22800_object26085:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.086951+0000", 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.20518761499999999, 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015377652, 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26086 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086951+0000", 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086951+0000", 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086952+0000", 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087009+0000", 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7194999999999997e-05 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087015+0000", 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6010000000000002e-06 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087019+0000", 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1760000000000003e-06 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.091114+0000", 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004095245 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.091272+0000", 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015817399999999999 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.091433+0000", 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016034299999999999 2024-04-18T22:23:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.099777+0000", 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083440709999999998 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102271+0000", 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024943550000000002 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102304+0000", 2024-04-18T22:23:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3308e-05 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102329+0000", 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4561000000000001e-05 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26099 5.f 5:fd0a425b:::benchmark_data_smithi032_22800_object26098:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.19981363999999999, 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01621906, 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26099 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092326+0000", 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.096e-06 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092379+0000", 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2737e-05 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092394+0000", 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4545999999999999e-05 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092399+0000", 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7320000000000003e-06 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.099921+0000", 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075215849999999999 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100044+0000", 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122786 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100132+0000", 2024-04-18T22:23:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8230000000000001e-05 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102949+0000", 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028173769999999998 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108443+0000", 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005493698 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108504+0000", 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1211999999999996e-05 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108544+0000", 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0061000000000002e-05 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26100 5.f 5:fb6fac53:::benchmark_data_smithi032_22800_object26099:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.199635743, 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016127091999999999, 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26100 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092504+0000", 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0299999999999997e-07 2024-04-18T22:23:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092632+0000", 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012751599999999999 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092644+0000", 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2618e-05 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092653+0000", 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-04-18T22:23:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100929+0000", 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082759610000000001 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101092+0000", 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016214300000000001 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101249+0000", 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015756099999999999 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.104207+0000", 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002957758 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108553+0000", 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043464280000000003 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108596+0000", 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3135e-05 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108630+0000", 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3798999999999999e-05 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:06.406 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:06.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:06.408 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:06.407+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:06.508 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:06.507+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:06.553 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:06.553 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:06.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.553 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:27249 5.f e26/23 5:ff07ac9a:::benchmark_data_smithi032_22800_object27248:head v 26'1639, mlcod=26'1637)", 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:06.552092+0000", 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002645321, 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002670764, 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552092+0000", 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552092+0000", 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552092+0000", 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2600000000000002e-07 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552190+0000", 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7208999999999997e-05 2024-04-18T22:23:06.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552202+0000", 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2394e-05 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552209+0000", 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4050000000000003e-06 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552261+0000", 2024-04-18T22:23:06.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1919999999999998e-05 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.552274+0000", 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.29e-05 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:27255 5.9 e26/23 5:9f0c9de5:::benchmark_data_smithi032_22800_object27254:head v 26'1751, mlcod=26'1749)", 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:06.553988+0000", 2024-04-18T22:23:06.557 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00074896499999999998, 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00084375200000000002, 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.553988+0000", 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.553988+0000", 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.553989+0000", 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1500000000000004e-07 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554097+0000", 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108175 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554108+0000", 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0739e-05 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554114+0000", 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6889999999999998e-06 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554164+0000", 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9475999999999999e-05 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554179+0000", 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5625e-05 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.561 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:27256 5.7 e26/23 5:e7c56091:::benchmark_data_smithi032_22800_object27255:head v 26'1730, mlcod=26'1728)", 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:06.553988+0000", 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00074899199999999997, 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00090543100000000005, 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.553988+0000", 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.553988+0000", 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.553989+0000", 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1930000000000001e-06 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554090+0000", 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100659 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554107+0000", 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6872e-05 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554121+0000", 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3902e-05 2024-04-18T22:23:06.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554185+0000", 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4105999999999999e-05 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554204+0000", 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9304999999999999e-05 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:27258 5.1 e26/23 5:8b04471f:::benchmark_data_smithi032_22800_object27257:head v 26'1638, mlcod=26'1636)", 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:06.554168+0000", 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00056914499999999996, 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00078439999999999998, 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554168+0000", 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554168+0000", 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554169+0000", 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5000000000000002e-07 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554215+0000", 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5871000000000002e-05 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554229+0000", 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4572e-05 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554239+0000", 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0037000000000001e-05 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554287+0000", 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7911000000000003e-05 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:06.554305+0000", 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7506000000000001e-05 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.569 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.570 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.570 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.570 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:06.570 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 4 2024-04-18T22:23:06.570 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:06.574 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:06.610 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:06.606+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:06.710 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:06.711+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:06.720 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:06.720 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:06.720 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:06.720 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:06.720 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:06.739 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:06.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:06.811 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:06.806+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:06.880 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:06.880 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.765010777000001, 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.881 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:06.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:06.886 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764942179000002, 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:06.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:06.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764846333000001, 2024-04-18T22:23:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764789450999999, 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764729779, 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764671817, 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764452124999998, 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764395196000002, 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764299309999998, 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764236982, 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764164504, 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.764012035999997, 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.763933313000003, 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.762412457000003, 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.762314105000002, 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.0805921519999999, 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.0799379389999999, 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.079831244, 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.077965335, 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.077300012, 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:06.986 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:06.988 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:06.988 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:06.987+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:06.989 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:06.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:06.999 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:07.089 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:07.086+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:07.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:07.190 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:07.188+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:07.190 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 11 16 28787 28771 163.455 152.625 0.00642277 0.00611204 2024-04-18T22:23:07.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:07.292+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:07.346 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:07.346 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:07.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:07.391 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:07.390+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:07.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:23:07.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:07.492+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:07.536 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:07.536 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:07.557 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:07.593 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:07.592+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:07.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:07.696+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:07.709 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:07.709 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:07.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:07.796 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:07.796+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:07.883 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:07.883 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:07.896 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:07.895+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:07.905 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:07.997 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:07.999+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:08.058 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:08.058 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:08.080 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:08.098 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:08.099+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:08.190 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 12 16 31257 31241 162.697 154.375 0.00802326 0.00614094 2024-04-18T22:23:08.198 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:08.199+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:08.232 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:08.232 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:08.252 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:08.299 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:08.300+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:08.394 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:08.395 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:08.400 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:08.398+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:08.413 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:08.442 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:23:08.501 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:08.498+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:08.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:08.604+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:08.607 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:08.607 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:08.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:08.703 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:08.702+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:08.792 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:08.792 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:08.805 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:08.803+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:08.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:08.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:08.904+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:09.006 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:09.007+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:09.016 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:09.017 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:09.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:09.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:09.106 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:09.108+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:09.181 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:09.181 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:09.190 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 13 16 33676 33660 161.811 151.188 0.00663431 0.00617466 2024-04-18T22:23:09.200 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:09.200 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:09.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:09.208+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:09.219 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:09.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.249 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.133281515999997, 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:09.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.133149023999998, 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:09.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:09.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.263 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.133077438000001, 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:09.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:09.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:09.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.132949185000001, 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:09.270 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:09.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:09.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.277 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.132839857999997, 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:09.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:09.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.284 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.132583521000001, 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:09.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:09.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.132496035000003, 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:09.290 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:09.294 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 37.132418049000002, 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:09.295 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:09.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:09.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4501309669999998, 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.300 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:09.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4484745879999998, 2024-04-18T22:23:09.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:09.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:09.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.448349871, 2024-04-18T22:23:09.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:09.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:09.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4465702130000002, 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4460875419999999, 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:09.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:09.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:09.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4458484670000002, 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.332 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:09.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:09.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:09.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.338 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4457276960000001, 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:09.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.442838284, 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:09.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:09.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.351 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4427164650000002, 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:09.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:09.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.442606016, 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.4425365480000001, 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:09.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.442479407, 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.369 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:09.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:09.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:09.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:09.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:09.374 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:09.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:09.376 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:09.374+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:09.396 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:23:09.396 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:09.396 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-04-18T22:23:09.396 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-18T22:23:09.396 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-18T22:23:09.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-04-18T22:23:09.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:09.476+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:09.534 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:34512 5.0 5:0e1b8217:::benchmark_data_smithi032_22800_object34511:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:09.535682+0000", 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00052508100000000005, 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054475300000000003, 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34512 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535682+0000", 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535682+0000", 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535683+0000", 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.75e-07 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535786+0000", 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010354299999999999 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535795+0000", 2024-04-18T22:23:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6179999999999995e-06 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535799+0000", 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4429999999999996e-06 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535846+0000", 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6239999999999998e-05 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535976+0000", 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013005100000000001 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536118+0000", 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142461 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:34514 5.8 5:16fc35b6:::benchmark_data_smithi032_22800_object34513:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:09.535981+0000", 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000226352, 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038529100000000001, 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34514 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535981+0000", 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535981+0000", 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.535982+0000", 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699999999999998e-07 2024-04-18T22:23:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536064+0000", 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2545000000000003e-05 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536070+0000", 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8680000000000004e-06 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536074+0000", 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9049999999999999e-06 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536106+0000", 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1776999999999997e-05 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536215+0000", 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109703 2024-04-18T22:23:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536343+0000", 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000127281 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4586.0:34498 5.d e27/19 ondisk, result = 0)", 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:09.536264+0000", 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9999433, 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182747, 2024-04-18T22:23:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536264+0000", 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536264+0000", 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536264+0000", 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536269+0000", 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6290000000000001e-06 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536279+0000", 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536286+0000", 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7320000000000001e-06 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536333+0000", 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7593000000000003e-05 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536341+0000", 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5310000000000004e-06 2024-04-18T22:23:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536447+0000", 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105818 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:34501 5.d 5:b788bbb5:::benchmark_data_smithi032_22800_object34500:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:09.532073+0000", 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0041338210000000002, 2024-04-18T22:23:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044446709999999999, 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34501 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.532073+0000", 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.532073+0000", 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.532074+0000", 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-04-18T22:23:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.532105+0000", 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1126e-05 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.532110+0000", 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2939999999999996e-06 2024-04-18T22:23:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.532114+0000", 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9600000000000002e-06 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.533673+0000", 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015584939999999999 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.533813+0000", 2024-04-18T22:23:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013961400000000001 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.533985+0000", 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172664 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.536504+0000", 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002519173 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2024-04-18T22:23:09.554 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:09.554 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:09.554 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-18T22:23:09.554 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:09.557 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:09.577 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:09.579+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:09.678 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:09.679+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:09.707 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:09.708 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:09.728 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:09.779 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:09.778+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:09.881 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:09.878+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:09.885 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:23:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.771099030999999, 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:23:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:23:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:23:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768765139000003, 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:23:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:23:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768662239999998, 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:23:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:23:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:23:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768612572000002, 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:23:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768546897, 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:23:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:23:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:23:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768487149999999, 2024-04-18T22:23:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768407080000003, 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:23:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:23:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:23:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:23:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768345672000002, 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:23:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.768261359, 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:23:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:23:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:23:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.761105626000003, 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25664 5.f 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.08260326, 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093405780999999993, 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25664 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802645+0000", 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013290900000000001 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802656+0000", 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-05 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802661+0000", 2024-04-18T22:23:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6729999999999997e-06 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802711+0000", 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9329000000000002e-05 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802804+0000", 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2929000000000004e-05 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802894+0000", 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0346999999999994e-05 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.807400+0000", 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045058290000000003 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895676+0000", 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088275705999999995 2024-04-18T22:23:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895810+0000", 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134635 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895917+0000", 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107309 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25666 5.f 5:f7add380:::benchmark_data_smithi032_22800_object25665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0824297429999996, 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093393754999999995, 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25666 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802686+0000", 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802716+0000", 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9954000000000002e-05 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802723+0000", 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5689999999999999e-06 2024-04-18T22:23:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802728+0000", 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6079999999999998e-06 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803716+0000", 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098778600000000005 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803871+0000", 2024-04-18T22:23:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015491100000000001 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.804071+0000", 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020055600000000001 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.810020+0000", 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059493339999999997 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895982+0000", 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085962083999999994 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896036+0000", 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3813999999999997e-05 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896079+0000", 2024-04-18T22:23:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2676000000000003e-05 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25673 5.d 5:becd3ab4:::benchmark_data_smithi032_22800_object25672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0770622269999999, 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085178675999999995, 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25673 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808054+0000", 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2900000000000002e-07 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808114+0000", 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9993e-05 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808126+0000", 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808136+0000", 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.064e-05 2024-04-18T22:23:09.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808186+0000", 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9471000000000001e-05 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808277+0000", 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0906e-05 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808387+0000", 2024-04-18T22:23:09.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011026099999999999 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.812706+0000", 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043185289999999998 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.892987+0000", 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080281292000000004 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893126+0000", 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013872099999999999 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893231+0000", 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010582300000000001 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25675 5.f 5:f60cbd32:::benchmark_data_smithi032_22800_object25674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:09.965 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0696206559999997, 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082348202999999995, 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25675 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815495+0000", 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815546+0000", 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0933999999999997e-05 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815572+0000", 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5936999999999999e-05 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815585+0000", 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2860000000000001e-05 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815664+0000", 2024-04-18T22:23:09.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9334999999999999e-05 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815838+0000", 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017410700000000001 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.816021+0000", 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018247599999999999 2024-04-18T22:23:09.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.817763+0000", 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001742494 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897556+0000", 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079792478999999999 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897717+0000", 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161199 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897843+0000", 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125415 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25682 5.b 5:d19b5078:::benchmark_data_smithi032_22800_object25681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9913032949999998, 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017001512999999999, 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25682 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4899999999999995e-07 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893846+0000", 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3546e-05 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893855+0000", 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5689999999999994e-06 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893862+0000", 2024-04-18T22:23:09.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7070000000000003e-06 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898400+0000", 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045379089999999997 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898589+0000", 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188965 2024-04-18T22:23:09.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898831+0000", 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000242181 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902115+0000", 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032839369999999998 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910719+0000", 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086036350000000001 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910771+0000", 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2584999999999999e-05 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910813+0000", 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1829999999999998e-05 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25693 5.e 5:715faa37:::benchmark_data_smithi032_22800_object25692:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:09.980 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9858950970000002, 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016385088999999999, 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25693 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899222+0000", 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.646e-06 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899274+0000", 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2642999999999997e-05 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899292+0000", 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8196e-05 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899304+0000", 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1866e-05 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899425+0000", 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121131 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899751+0000", 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032516000000000001 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.900183+0000", 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043271999999999999 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902912+0000", 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027285930000000001 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915375+0000", 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012463093 2024-04-18T22:23:09.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915503+0000", 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012793099999999999 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915605+0000", 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010211 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26085 5.f 5:ffd60b63:::benchmark_data_smithi032_22800_object26084:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.086875+0000", 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7982395420000001, 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015390157, 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26085 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.987 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086875+0000", 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086875+0000", 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086876+0000", 2024-04-18T22:23:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2900000000000003e-07 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086922+0000", 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6449000000000002e-05 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086934+0000", 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1911e-05 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086939+0000", 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4730000000000002e-06 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.086998+0000", 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9401999999999998e-05 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087212+0000", 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021393700000000001 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.087470+0000", 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025744800000000002 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092867+0000", 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053969029999999998 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102165+0000", 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092985849999999998 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102212+0000", 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6807000000000001e-05 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102266+0000", 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3613000000000003e-05 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26099 5.f 5:fd0a425b:::benchmark_data_smithi032_22800_object26098:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7927896919999999, 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01621906, 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26099 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:09.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092326+0000", 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.096e-06 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092379+0000", 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2737e-05 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092394+0000", 2024-04-18T22:23:09.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4545999999999999e-05 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092399+0000", 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7320000000000003e-06 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.099921+0000", 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075215849999999999 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100044+0000", 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122786 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100132+0000", 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8230000000000001e-05 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102949+0000", 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028173769999999998 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108443+0000", 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005493698 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108504+0000", 2024-04-18T22:23:09.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1211999999999996e-05 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108544+0000", 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0061000000000002e-05 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26100 5.f 5:fb6fac53:::benchmark_data_smithi032_22800_object26099:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.792611795, 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016127091999999999, 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26100 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.000 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092504+0000", 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0299999999999997e-07 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092632+0000", 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012751599999999999 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:10.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092644+0000", 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2618e-05 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092653+0000", 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100929+0000", 2024-04-18T22:23:10.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082759610000000001 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101092+0000", 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016214300000000001 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101249+0000", 2024-04-18T22:23:10.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015756099999999999 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.104207+0000", 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002957758 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108553+0000", 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043464280000000003 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108596+0000", 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3135e-05 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108630+0000", 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3798999999999999e-05 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:10.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26931 5.b 5:d9657119:::benchmark_data_smithi032_22800_object26930:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.4651487809999999, 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016342974999999999, 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:10.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26931 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:10.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419967+0000", 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420067+0000", 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9531999999999995e-05 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420075+0000", 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420082+0000", 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421326+0000", 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012431720000000001 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421410+0000", 2024-04-18T22:23:10.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4254999999999999e-05 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421491+0000", 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0814000000000002e-05 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.422721+0000", 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012302109999999999 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436212+0000", 2024-04-18T22:23:10.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013491385999999999 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436268+0000", 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5585999999999999e-05 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436309+0000", 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1374000000000001e-05 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:10.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:10.014 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:10.015 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:10.015+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:10.016 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:10.116 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:10.115+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:10.162 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36041 5.9 e27/23 5:9e134d5c:::benchmark_data_smithi032_22800_object36040:head v 27'2263, mlcod=27'2261)", 2024-04-18T22:23:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.155510+0000", 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0062040410000000004, 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0062530889999999999, 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155510+0000", 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155510+0000", 2024-04-18T22:23:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155511+0000", 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0399999999999995e-07 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155573+0000", 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2569000000000004e-05 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155584+0000", 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1177e-05 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155592+0000", 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8140000000000001e-06 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155642+0000", 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9515000000000003e-05 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155661+0000", 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8879e-05 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36042 5.c e27/19 5:37a8dd8b:::benchmark_data_smithi032_22800_object36041:head v 27'2267, mlcod=27'2265)", 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.155722+0000", 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0059918539999999996, 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061964510000000004, 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155722+0000", 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155722+0000", 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155723+0000", 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5600000000000005e-07 2024-04-18T22:23:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155768+0000", 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5173000000000003e-05 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155780+0000", 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2486999999999999e-05 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155788+0000", 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0909999999999993e-06 2024-04-18T22:23:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155844+0000", 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5584999999999997e-05 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.155863+0000", 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9239999999999999e-05 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.169 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36046 5.d e27/19 5:bd8090ee:::benchmark_data_smithi032_22800_object36045:head v 27'2266, mlcod=27'2264)", 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.157444+0000", 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0042704140000000002, 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046212069999999996, 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157444+0000", 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157444+0000", 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157444+0000", 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2300000000000001e-07 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157636+0000", 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000191936 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157659+0000", 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3261e-05 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157670+0000", 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0936e-05 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157718+0000", 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7651000000000002e-05 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157730+0000", 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1977999999999999e-05 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36043 5.c e27/19 5:31fb4406:::benchmark_data_smithi032_22800_object36042:head v 27'2268, mlcod=27'2265)", 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.157513+0000", 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0042005549999999999, 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046939950000000003, 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.173 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157513+0000", 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157513+0000", 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157514+0000", 2024-04-18T22:23:10.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6100000000000002e-07 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157661+0000", 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147529 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157673+0000", 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1721e-05 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157679+0000", 2024-04-18T22:23:10.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3770000000000001e-06 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157726+0000", 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6403999999999998e-05 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.157748+0000", 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2204999999999998e-05 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.162184+0000", 2024-04-18T22:23:10.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044361390000000004 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.162215+0000", 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1168999999999999e-05 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36048 5.c e27/19 5:3387c9cc:::benchmark_data_smithi032_22800_object36047:head v 27'2269, mlcod=27'2265)", 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.158731+0000", 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0029828810000000002, 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003703146, 2024-04-18T22:23:10.177 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158731+0000", 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158731+0000", 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158732+0000", 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6300000000000005e-07 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158801+0000", 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8943999999999996e-05 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158810+0000", 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1269999999999992e-06 2024-04-18T22:23:10.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158814+0000", 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.775e-06 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158876+0000", 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1472999999999999e-05 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158895+0000", 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.916e-05 2024-04-18T22:23:10.180 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36049 5.1 e27/23 5:830fb7cb:::benchmark_data_smithi032_22800_object36048:head v 27'2188, mlcod=27'2185)", 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.158875+0000", 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0028391459999999999, 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037004920000000001, 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158875+0000", 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158875+0000", 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158875+0000", 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6600000000000002e-07 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158964+0000", 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8317000000000002e-05 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158969+0000", 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.429e-06 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.158973+0000", 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5870000000000001e-06 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159032+0000", 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9159000000000001e-05 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159056+0000", 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3818000000000001e-05 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36050 5.9 e27/23 5:9df7ec5a:::benchmark_data_smithi032_22800_object36049:head v 27'2265, mlcod=27'2261)", 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.159021+0000", 2024-04-18T22:23:10.184 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0026925550000000001, 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037038599999999998, 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159021+0000", 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159021+0000", 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159022+0000", 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1800000000000005e-07 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159159+0000", 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000136995 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159171+0000", 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2167999999999999e-05 2024-04-18T22:23:10.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159179+0000", 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5050000000000001e-06 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159263+0000", 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4005000000000006e-05 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.159296+0000", 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3288999999999999e-05 2024-04-18T22:23:10.187 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36052 5.6 e27/19 5:6ece85b9:::benchmark_data_smithi032_22800_object36051:head v 27'2203, mlcod=27'2201)", 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.160599+0000", 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001114741, 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023035629999999998, 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160599+0000", 2024-04-18T22:23:10.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160599+0000", 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160600+0000", 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9400000000000005e-07 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160645+0000", 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5584000000000003e-05 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160655+0000", 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6309999999999998e-06 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160662+0000", 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8220000000000001e-06 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160705+0000", 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2769000000000003e-05 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160716+0000", 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1446e-05 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36053 5.f e27/23 5:f1dc5cdd:::benchmark_data_smithi032_22800_object36052:head v 27'2192, mlcod=27'2190)", 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.160695+0000", 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0010185789999999999, 2024-04-18T22:23:10.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023634419999999999, 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160695+0000", 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160695+0000", 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160696+0000", 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4099999999999998e-07 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160806+0000", 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000109972 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160816+0000", 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6749999999999997e-06 2024-04-18T22:23:10.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160826+0000", 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0141e-05 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160913+0000", 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6947999999999995e-05 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160940+0000", 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7064999999999999e-05 2024-04-18T22:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36051 5.c e27/19 5:3c620964:::benchmark_data_smithi032_22800_object36050:head v 27'2270, mlcod=27'2265)", 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.160709+0000", 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0010053849999999999, 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025221919999999999, 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160709+0000", 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160709+0000", 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160709+0000", 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9999999999999999e-07 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160804+0000", 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5439999999999994e-05 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160820+0000", 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5531000000000001e-05 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160826+0000", 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2380000000000002e-06 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160881+0000", 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5291000000000003e-05 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.160897+0000", 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5899000000000001e-05 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36056 5.d e27/19 5:b944c731:::benchmark_data_smithi032_22800_object36055:head v 27'2267, mlcod=27'2264)", 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.161363+0000", 2024-04-18T22:23:10.198 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00035099399999999998, 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001946064, 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161363+0000", 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161363+0000", 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161363+0000", 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1100000000000001e-07 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161420+0000", 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6762000000000003e-05 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161427+0000", 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6320000000000002e-06 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161430+0000", 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.658e-06 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161469+0000", 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8447e-05 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.161482+0000", 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3487e-05 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.201 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:36057 5.2 e27/23 5:4818f566:::benchmark_data_smithi032_22800_object36056:head v 27'2233, mlcod=27'2231)", 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:10.163083+0000", 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.998631, 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030255899999999999, 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163083+0000", 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163083+0000", 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163083+0000", 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2200000000000004e-07 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163149+0000", 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6018000000000004e-05 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163162+0000", 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2938999999999999e-05 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163168+0000", 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3530000000000002e-06 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163219+0000", 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0936000000000002e-05 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:10.163239+0000", 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0480999999999999e-05 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 12 2024-04-18T22:23:10.205 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:10.206 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 14 16 36140 36124 161.252 154 0.00373628 0.00619644 2024-04-18T22:23:10.207 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:10.217 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:10.215+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:10.317 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:10.319+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:10.350 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:10.350 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:10.350 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:10.350 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:10.350 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:10.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:10.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:10.420+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:10.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:23:10.520 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:10.520+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:10.523 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408683623999998, 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.524 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:10.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:10.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408615026, 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:10.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:10.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408519179999999, 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.536 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:10.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:10.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.541 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408462298000003, 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:10.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:10.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408402625999997, 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.547 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:10.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.552 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408344663999998, 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:10.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408124972000003, 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.408068043, 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.407972157000003, 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.407909828999998, 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.407837350999998, 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.407684883000002, 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.40760616, 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.406085304000001, 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.405986951999999, 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.7242649989999999, 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.7236107860000001, 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.7235040909999997, 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.7216381819999995, 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:10.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:10.624 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4.7209728589999997, 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:10.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:10.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:10.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:10.629 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:10.631 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:10.632+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:10.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:10.732 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:10.732+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37578 5.c 5:3f21291f:::benchmark_data_smithi032_22800_object37577:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.780492+0000", 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0046300509999999996, 2024-04-18T22:23:10.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046580759999999997, 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37578 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780492+0000", 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780492+0000", 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780493+0000", 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0610000000000001e-06 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780526+0000", 2024-04-18T22:23:10.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3343000000000002e-05 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780538+0000", 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1986e-05 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780546+0000", 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5480000000000004e-06 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780609+0000", 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2978999999999995e-05 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780724+0000", 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115771 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780858+0000", 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013354 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782470+0000", 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001612451 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37579 5.7 5:e3833798:::benchmark_data_smithi032_22800_object37578:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.780564+0000", 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0045576159999999996, 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046700049999999996, 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37579 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780564+0000", 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780564+0000", 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780565+0000", 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.68e-07 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780598+0000", 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3584999999999998e-05 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780608+0000", 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.679e-06 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780614+0000", 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4030000000000004e-06 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780681+0000", 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6853999999999994e-05 2024-04-18T22:23:10.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780849+0000", 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016729599999999999 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.781035+0000", 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000185942 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782880+0000", 2024-04-18T22:23:10.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018459430000000001 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:37581 5.8 e27/19 5:1547fcc4:::benchmark_data_smithi032_22800_object37580:head v 27'2335, mlcod=27'2333)", 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.780627+0000", 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0044951909999999999, 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004681307, 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780627+0000", 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780627+0000", 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780627+0000", 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2399999999999998e-07 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780719+0000", 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1553999999999998e-05 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780727+0000", 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4279999999999996e-06 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780734+0000", 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0079999999999996e-06 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780802+0000", 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7727999999999999e-05 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.780820+0000", 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8573999999999999e-05 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37580 5.c 5:3aaa53f7:::benchmark_data_smithi032_22800_object37579:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.798 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.782395+0000", 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002727017, 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029735, 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37580 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782395+0000", 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782395+0000", 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782396+0000", 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9899999999999999e-07 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782447+0000", 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1842999999999998e-05 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782459+0000", 2024-04-18T22:23:10.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1545e-05 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782467+0000", 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2789999999999998e-06 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782520+0000", 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2298e-05 2024-04-18T22:23:10.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782696+0000", 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017621800000000001 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782844+0000", 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014835400000000001 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37582 5.2 5:4cd6597f:::benchmark_data_smithi032_22800_object37581:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.782485+0000", 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026365099999999999, 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002964005, 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.804 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37582 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782485+0000", 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782485+0000", 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782486+0000", 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4300000000000003e-07 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782534+0000", 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7787000000000002e-05 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782553+0000", 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8885999999999998e-05 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782559+0000", 2024-04-18T22:23:10.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7769999999999997e-06 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782619+0000", 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9771000000000001e-05 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782756+0000", 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000137393 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782841+0000", 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4176000000000002e-05 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37583 5.2 5:445885e8:::benchmark_data_smithi032_22800_object37582:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.809 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.782577+0000", 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025448269999999999, 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029505400000000002, 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37583 2024-04-18T22:23:10.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782577+0000", 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782577+0000", 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782578+0000", 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0500000000000002e-07 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782622+0000", 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4542000000000001e-05 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782630+0000", 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5850000000000002e-06 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782636+0000", 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7479999999999999e-06 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784073+0000", 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001436859 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784187+0000", 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114879 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784333+0000", 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014558999999999999 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37586 5.3 5:c379cb37:::benchmark_data_smithi032_22800_object37585:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.782650+0000", 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0024722979999999999, 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029607219999999998, 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37586 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782650+0000", 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782650+0000", 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782650+0000", 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.68e-07 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782676+0000", 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6211e-05 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782683+0000", 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0119999999999999e-06 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782688+0000", 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0300000000000001e-06 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783928+0000", 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012400099999999999 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784065+0000", 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013666000000000001 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784202+0000", 2024-04-18T22:23:10.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000137441 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:37585 5.b e27/19 5:db34fbab:::benchmark_data_smithi032_22800_object37584:head v 27'2398, mlcod=27'2396)", 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.782635+0000", 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0024870270000000002, 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030419420000000002, 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782635+0000", 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782635+0000", 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782636+0000", 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6599999999999995e-07 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782708+0000", 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.292e-05 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782718+0000", 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5089999999999999e-06 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.782724+0000", 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4309999999999999e-06 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783844+0000", 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011196800000000001 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783864+0000", 2024-04-18T22:23:10.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0386000000000001e-05 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37588 5.5 5:aec27830:::benchmark_data_smithi032_22800_object37587:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.783018+0000", 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021035770000000001, 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027176549999999998, 2024-04-18T22:23:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37588 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783018+0000", 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783018+0000", 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783019+0000", 2024-04-18T22:23:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5799999999999998e-07 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783157+0000", 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138071 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783168+0000", 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0865e-05 2024-04-18T22:23:10.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783175+0000", 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1590000000000004e-06 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783223+0000", 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7932000000000001e-05 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783318+0000", 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4605999999999995e-05 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783432+0000", 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114722 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:10.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.785669+0000", 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022363439999999999 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:37587 5.8 e27/19 5:1a04b070:::benchmark_data_smithi032_22800_object37586:head v 27'2336, mlcod=27'2333)", 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.783345+0000", 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001777215, 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024690850000000002, 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783345+0000", 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783345+0000", 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783345+0000", 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6000000000000004e-07 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783426+0000", 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0531999999999994e-05 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783436+0000", 2024-04-18T22:23:10.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9760000000000007e-06 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783442+0000", 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0990000000000004e-06 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783929+0000", 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00048742300000000001 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783946+0000", 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7266999999999999e-05 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:37589 5.4 e27/19 5:233f235d:::benchmark_data_smithi032_22800_object37588:head v 27'2412, mlcod=27'2410)", 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.783614+0000", 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001507806, 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022528880000000002, 2024-04-18T22:23:10.833 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783614+0000", 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783614+0000", 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783615+0000", 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.06e-07 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783672+0000", 2024-04-18T22:23:10.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7484000000000001e-05 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783679+0000", 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4000000000000003e-06 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.783684+0000", 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0359999999999997e-06 2024-04-18T22:23:10.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784472+0000", 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00078732200000000002 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784496+0000", 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4028000000000001e-05 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:37590 5.a 5:5a0dc947:::benchmark_data_smithi032_22800_object37589:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:10.784106+0000", 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010161829999999999, 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001819914, 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:10.838 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 37590 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784106+0000", 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:10.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784106+0000", 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784106+0000", 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5199999999999995e-07 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784192+0000", 2024-04-18T22:23:10.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5322999999999996e-05 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784202+0000", 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8519999999999999e-06 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784208+0000", 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5470000000000002e-06 2024-04-18T22:23:10.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784256+0000", 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7441000000000002e-05 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784368+0000", 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111875 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:10.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:10.784483+0000", 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115813 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 12 2024-04-18T22:23:10.843 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:10.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:10.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:10.843+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:10.852 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:23:10.946 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:10.942+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:11.001 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:11.001 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:11.001 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:11.001 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:11.001 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:11.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:11.048 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:11.042+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:11.149 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:11.146+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:11.176 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:11.176 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:11.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:11.176 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.060331298999998, 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:11.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:11.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.060198806999999, 2024-04-18T22:23:11.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:11.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:11.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.060127221000002, 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:11.190 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:11.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:11.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:11.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:11.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.059998968000002, 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:11.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:11.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.059889640999998, 2024-04-18T22:23:11.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:11.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.218 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.059633304000002, 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:11.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.059545817999997, 2024-04-18T22:23:11.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:11.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.229 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.059467832000003, 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:11.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:11.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:11.235 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.37718075, 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:11.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:11.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.375524371, 2024-04-18T22:23:11.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:11.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:11.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:11.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3753996539999997, 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:11.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:11.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3736199960000004, 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3731373250000001, 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3728982500000004, 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3727774789999998, 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3698880669999998, 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3697662480000004, 2024-04-18T22:23:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3696557990000002, 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3695863309999998, 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.3695291899999997, 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:11.304 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:11.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:11.305 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 15 16 38571 38555 160.63 151.938 0.00803187 0.0062203 2024-04-18T22:23:11.306 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:11.307 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:11.307+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:11.406 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:11.402+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4586.0:39166 5.a e28/19 5:52fd06b6:::benchmark_data_smithi032_22800_object39165:head v 28'2440, mlcod=28'2436)", 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.455883+0000", 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0056537330000000002, 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00567681, 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455883+0000", 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455883+0000", 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455884+0000", 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1200000000000003e-07 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455913+0000", 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9450000000000001e-05 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455922+0000", 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8859999999999993e-06 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455930+0000", 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3119999999999996e-06 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455975+0000", 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5411999999999998e-05 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455996+0000", 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0743000000000001e-05 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39169 5.4 5:241dafa0:::benchmark_data_smithi032_22800_object39168:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.457312+0000", 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0042247839999999997, 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043150439999999997, 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39169 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457312+0000", 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457312+0000", 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457313+0000", 2024-04-18T22:23:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3799999999999996e-07 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457378+0000", 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4848e-05 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457412+0000", 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4282000000000001e-05 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457419+0000", 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5950000000000004e-06 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457459+0000", 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0402999999999999e-05 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457568+0000", 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010912999999999999 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457668+0000", 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9401999999999995e-05 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460487+0000", 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028191829999999998 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39170 5.9 5:9fc58bb7:::benchmark_data_smithi032_22800_object39169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.457432+0000", 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0041055070000000004, 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00427546, 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39170 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457432+0000", 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457432+0000", 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457432+0000", 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457466+0000", 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4007000000000002e-05 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457474+0000", 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457480+0000", 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.913e-06 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457541+0000", 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0551999999999999e-05 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457674+0000", 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133251 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457814+0000", 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000140051 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39171 5.b 5:df147bb4:::benchmark_data_smithi032_22800_object39170:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.457492+0000", 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.004045356, 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042874749999999998, 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39171 2024-04-18T22:23:11.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457492+0000", 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457492+0000", 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457492+0000", 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457520+0000", 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7387999999999999e-05 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457525+0000", 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9949999999999996e-06 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457539+0000", 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4149e-05 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457632+0000", 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2992999999999995e-05 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457758+0000", 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012568900000000001 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.457884+0000", 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012634500000000001 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461175+0000", 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032909269999999999 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.478 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39172 5.b 5:d76c7acf:::benchmark_data_smithi032_22800_object39171:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.458164+0000", 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.003373296, 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036928579999999998, 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39172 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458164+0000", 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458164+0000", 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458165+0000", 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458284+0000", 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011927899999999999 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458292+0000", 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6799999999999993e-06 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458297+0000", 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7250000000000002e-06 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459214+0000", 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091639800000000002 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459420+0000", 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020666900000000001 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459563+0000", 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142743 2024-04-18T22:23:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39174 5.9 5:9cc7caea:::benchmark_data_smithi032_22800_object39173:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.458511+0000", 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0030267219999999999, 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034173889999999998, 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39174 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458511+0000", 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458511+0000", 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458511+0000", 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458588+0000", 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6767999999999994e-05 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458596+0000", 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1189999999999997e-06 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.458600+0000", 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4469999999999999e-06 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459193+0000", 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059236 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459334+0000", 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014130600000000001 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459481+0000", 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014689400000000001 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39175 5.e 5:7f7c6657:::benchmark_data_smithi032_22800_object39174:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.459908+0000", 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00162963, 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020984799999999998, 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39175 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459908+0000", 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459908+0000", 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459908+0000", 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000003e-07 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459961+0000", 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3254000000000002e-05 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459970+0000", 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2719999999999997e-06 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.459974+0000", 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.651e-06 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460014+0000", 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9782e-05 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460126+0000", 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111582 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460218+0000", 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2625000000000002e-05 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39177 5.d 5:babb5d81:::benchmark_data_smithi032_22800_object39176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.460890+0000", 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00064681299999999999, 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011880790000000001, 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39177 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460890+0000", 2024-04-18T22:23:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460890+0000", 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460891+0000", 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3099999999999996e-07 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460962+0000", 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0664e-05 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460972+0000", 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0137e-05 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460979+0000", 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0280000000000003e-06 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461033+0000", 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3488e-05 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461181+0000", 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014829399999999999 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461318+0000", 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136806 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.495 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39178 5.8 5:1fe6af67:::benchmark_data_smithi032_22800_object39177:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.460994+0000", 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00054327900000000003, 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00115621, 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39178 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460994+0000", 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460994+0000", 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.460995+0000", 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000001e-07 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461029+0000", 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4113999999999999e-05 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461036+0000", 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1550000000000001e-06 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461050+0000", 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4279e-05 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461095+0000", 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.494e-05 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461171+0000", 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6038999999999995e-05 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461251+0000", 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9991e-05 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39165 5.b 5:d82fb931:::benchmark_data_smithi032_22800_object39164:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.455286+0000", 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0062515360000000002, 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069363699999999999, 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39165 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455286+0000", 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455286+0000", 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455286+0000", 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455355+0000", 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9034000000000003e-05 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455363+0000", 2024-04-18T22:23:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0590000000000003e-06 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455368+0000", 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2069999999999997e-06 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455428+0000", 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0409999999999999e-05 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455574+0000", 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014581800000000001 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455721+0000", 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014673299999999999 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.461127+0000", 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054067949999999998 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:39167 5.f 5:f6ede71b:::benchmark_data_smithi032_22800_object39166:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:11.455778+0000", 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0057589599999999996, 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0065359659999999998, 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39167 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455778+0000", 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455778+0000", 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455779+0000", 2024-04-18T22:23:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455841+0000", 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2086000000000002e-05 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455850+0000", 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8380000000000008e-06 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455854+0000", 2024-04-18T22:23:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3610000000000003e-06 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.455899+0000", 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4539999999999997e-05 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.456016+0000", 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117477 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.456095+0000", 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9229999999999999e-05 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:11.462219+0000", 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061236939999999998 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 11 2024-04-18T22:23:11.509 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:11.510 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:11.511 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:11.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:11.512 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:11.506+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:11.513 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:11.611 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:11.611+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:11.710 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:11.710 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:11.712 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:11.711+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:11.731 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:11.813 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:11.815+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:11.887 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:11.888 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:11.909 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:11.914 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:11.910+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:12.015 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.015+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.060 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:12.060 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:12.079 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:12.122 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.123+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 16 16 40906 40890 159.71 145.938 0.00785056 0.00625564 2024-04-18T22:23:12.217 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:12.215+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:12.265 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:12.265 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:12.284 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:12.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.331+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.419+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.429 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:12.429 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:12.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:12.520 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:12.518+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:12.599 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:12.599 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:12.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:12.620 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.619+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.721 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:12.718+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:12.770 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:12.770 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:12.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:12.824 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.823+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:23:12.923 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:12.923+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:12.938 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:12.938 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:12.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:13.024 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:13.022+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:13.125 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:13.123+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:13.125 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:13.125 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:13.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:13.190 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 17 15 43179 43164 158.675 142.125 0.00564503 0.00629717 2024-04-18T22:23:13.444 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:13.444 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:13.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:13.445 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:13.443+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:13.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:13.544 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:13.543+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:13.597 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:13.597 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:13.619 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:13.644 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:13.642+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:13.700 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:13.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:13.718 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:13.739 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:13.740 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:13.745 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:13.743+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:13.762 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:13.762 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:13.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.762 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:44518 5.5 e28/19 5:ac4fc298:::benchmark_data_smithi032_22800_object44517:head v 28'2688, mlcod=28'2683)", 2024-04-18T22:23:13.762 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:13.762915+0000", 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001068212, 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001088108, 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.762915+0000", 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:13.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.762915+0000", 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.762915+0000", 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.37e-07 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.762960+0000", 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5237000000000001e-05 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.762977+0000", 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6416999999999999e-05 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.762987+0000", 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8430000000000005e-06 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763038+0000", 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1057000000000003e-05 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763060+0000", 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2310000000000002e-05 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:44523 5.c e28/19 5:3f91a852:::benchmark_data_smithi032_22800_object44522:head v 28'2755, mlcod=28'2753)", 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:13.763464+0000", 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00051876299999999999, 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00058625599999999995, 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763464+0000", 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763464+0000", 2024-04-18T22:23:13.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763465+0000", 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4600000000000005e-07 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763546+0000", 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1065999999999999e-05 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763555+0000", 2024-04-18T22:23:13.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2310000000000002e-06 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763561+0000", 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1140000000000002e-06 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763612+0000", 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1211999999999997e-05 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:13.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763633+0000", 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0571e-05 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:44521 5.d e28/19 5:b23618cb:::benchmark_data_smithi032_22800_object44520:head v 28'2827, mlcod=28'2825)", 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:13.763566+0000", 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00041689100000000002, 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00053330200000000004, 2024-04-18T22:23:13.770 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763566+0000", 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763566+0000", 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763566+0000", 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7999999999999995e-07 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763639+0000", 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2800999999999996e-05 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763650+0000", 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0463e-05 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763656+0000", 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6930000000000001e-06 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763702+0000", 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5333000000000001e-05 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:13.763721+0000", 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8893999999999999e-05 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:13.774 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:13.774 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 3 2024-04-18T22:23:13.774 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:13.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:13.846 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:13.842+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:23:13.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:13.947+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:13.979 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:13.979 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:13.979 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:13.979 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:13.979 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:14.000 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:14.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:14.087+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:14.149 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:14.147+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:14.189 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:14.189 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073596674999997, 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.190 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:14.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:14.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:14.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:14.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:14.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073528076999999, 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.198 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:14.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:14.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:14.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:14.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:14.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.205 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073432230999998, 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:14.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.212 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073375349000003, 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:14.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:14.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.218 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073315676999997, 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:14.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:14.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:14.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073257714999997, 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:14.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:14.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.235 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.073038023000002, 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:14.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:14.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.241 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.072981093999999, 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.243 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:14.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:14.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.072885208000002, 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:14.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:14.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:14.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.072822879999997, 2024-04-18T22:23:14.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:14.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:14.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:14.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.072750401999997, 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:14.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:14.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:14.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.072597934000001, 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:14.265 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:14.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:14.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:14.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.072519210999999, 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:14.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:14.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:14.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.070998355, 2024-04-18T22:23:14.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:14.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 42.070900002999998, 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:14.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:14.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.285 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 8.3891780499999999, 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 8.3885238369999993, 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.290 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 8.3884171419999998, 2024-04-18T22:23:14.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:14.298 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 8.3865512330000005, 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:14.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:14.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 8.3858859100000007, 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.306 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.307 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.307 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:14.307 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:14.307 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:14.308 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 18 16 45491 45475 157.883 144.438 0.0102048 0.0063284 2024-04-18T22:23:14.308 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:14.307+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:14.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:14.409 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:14.406+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:14.459 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46089 5.6 5:6e175472:::benchmark_data_smithi032_22800_object46088:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.453774+0000", 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0066747400000000002, 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066943100000000002, 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46089 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453774+0000", 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453774+0000", 2024-04-18T22:23:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453775+0000", 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453838+0000", 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2476000000000003e-05 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453848+0000", 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9420000000000007e-06 2024-04-18T22:23:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453856+0000", 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3329999999999997e-06 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454438+0000", 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000582293 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454557+0000", 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118785 2024-04-18T22:23:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454680+0000", 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122607 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46090 5.3 5:ca30132a:::benchmark_data_smithi032_22800_object46089:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.453871+0000", 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0065783819999999998, 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066477150000000002, 2024-04-18T22:23:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46090 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453871+0000", 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453871+0000", 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453871+0000", 2024-04-18T22:23:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4799999999999998e-07 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453902+0000", 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1151e-05 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453910+0000", 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8310000000000001e-06 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453916+0000", 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0519999999999998e-06 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.455333+0000", 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014172639999999999 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.455521+0000", 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018752499999999999 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.455597+0000", 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6259000000000003e-05 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:46088 5.8 e29/19 5:1bb8a762:::benchmark_data_smithi032_22800_object46087:head v 29'2865, mlcod=29'2863)", 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.454117+0000", 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0063322700000000001, 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064429520000000001, 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454117+0000", 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454117+0000", 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454117+0000", 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454181+0000", 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3590999999999995e-05 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454192+0000", 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0886e-05 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454198+0000", 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8470000000000001e-06 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454719+0000", 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00052160599999999998 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454738+0000", 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8408000000000001e-05 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46092 5.5 5:a65bf1b0:::benchmark_data_smithi032_22800_object46091:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.454708+0000", 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0057411340000000002, 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005886283, 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46092 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454708+0000", 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454708+0000", 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454709+0000", 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3900000000000004e-07 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454782+0000", 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3102999999999994e-05 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454791+0000", 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0270000000000001e-06 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454798+0000", 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1570000000000003e-06 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454850+0000", 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1666000000000003e-05 2024-04-18T22:23:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.454967+0000", 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117046 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.455088+0000", 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012126200000000001 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:46093 5.e e29/19 5:7b2ca283:::benchmark_data_smithi032_22800_object46092:head v 29'2787, mlcod=29'2784)", 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.456357+0000", 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0040923929999999997, 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042892140000000004, 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456357+0000", 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456357+0000", 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456357+0000", 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4000000000000002e-07 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456447+0000", 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9664000000000001e-05 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456463+0000", 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6371e-05 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456469+0000", 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1959999999999996e-06 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456519+0000", 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9898999999999998e-05 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456533+0000", 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.418e-05 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46094 5.c 5:35cfdddf:::benchmark_data_smithi032_22800_object46093:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.456726+0000", 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0037228040000000001, 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039783539999999999, 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46094 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456726+0000", 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456726+0000", 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456727+0000", 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5799999999999998e-07 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456819+0000", 2024-04-18T22:23:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2120999999999994e-05 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456828+0000", 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6750000000000008e-06 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456832+0000", 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7520000000000004e-06 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456888+0000", 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5203e-05 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457043+0000", 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000155291 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457183+0000", 2024-04-18T22:23:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140426 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46095 5.6 5:660f185e:::benchmark_data_smithi032_22800_object46094:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.456848+0000", 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0036013579999999998, 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039339509999999998, 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46095 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456848+0000", 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456848+0000", 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456848+0000", 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0999999999999999e-07 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456880+0000", 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.154e-05 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456885+0000", 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3890000000000004e-06 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456889+0000", 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0010000000000003e-06 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.456937+0000", 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8053999999999998e-05 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457063+0000", 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012586100000000001 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457185+0000", 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121616 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46096 5.1 5:8a0ec3a7:::benchmark_data_smithi032_22800_object46095:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.457441+0000", 2024-04-18T22:23:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030082540000000001, 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034157390000000001, 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46096 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457441+0000", 2024-04-18T22:23:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457441+0000", 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457441+0000", 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9599999999999999e-07 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457554+0000", 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112763 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457561+0000", 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-06 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457565+0000", 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9559999999999999e-06 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457605+0000", 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9740000000000002e-05 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457689+0000", 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4377999999999997e-05 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.457765+0000", 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5877999999999994e-05 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46099 5.3 5:c644bb8c:::benchmark_data_smithi032_22800_object46098:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.459280+0000", 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0011688289999999999, 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001651104, 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46099 2024-04-18T22:23:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459280+0000", 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459280+0000", 2024-04-18T22:23:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459281+0000", 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.018e-06 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459346+0000", 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4927999999999999e-05 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459355+0000", 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1419999999999999e-06 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459360+0000", 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8189999999999999e-06 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459406+0000", 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5521e-05 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459492+0000", 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6714999999999999e-05 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459573+0000", 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0791000000000006e-05 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46101 5.5 5:a06ce1d2:::benchmark_data_smithi032_22800_object46100:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.459500+0000", 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00094941700000000003, 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015100630000000001, 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46101 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459500+0000", 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459500+0000", 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459500+0000", 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.46e-07 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459637+0000", 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000137214 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459643+0000", 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7459999999999997e-06 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459647+0000", 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9910000000000004e-06 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459682+0000", 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5023e-05 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459761+0000", 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9202999999999994e-05 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459840+0000", 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8836000000000002e-05 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:46098 5.0 e29/19 5:0def4e60:::benchmark_data_smithi032_22800_object46097:head v 29'2952, mlcod=29'2949)", 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.459967+0000", 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00048163500000000001, 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011198829999999999, 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459967+0000", 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459967+0000", 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.459968+0000", 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4700000000000001e-07 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460027+0000", 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8990000000000003e-05 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460036+0000", 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0839999999999998e-06 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460043+0000", 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.691e-06 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460104+0000", 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1515999999999999e-05 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460130+0000", 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5613999999999999e-05 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:46102 5.8 e29/19 5:1f3b046c:::benchmark_data_smithi032_22800_object46101:head v 29'2866, mlcod=29'2863)", 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.460066+0000", 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00038298, 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010804359999999999, 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460066+0000", 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460066+0000", 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460066+0000", 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9099999999999999e-07 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460108+0000", 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1906999999999997e-05 2024-04-18T22:23:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460117+0000", 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2930000000000001e-06 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460126+0000", 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7200000000000001e-06 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460166+0000", 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9232e-05 2024-04-18T22:23:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460182+0000", 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6079999999999999e-05 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:46100 5.b e29/19)", 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.460138+0000", 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00031089000000000003, 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00106554, 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460138+0000", 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460138+0000", 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460138+0000", 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0199999999999998e-07 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460172+0000", 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3492999999999999e-05 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460179+0000", 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.460185+0000", 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2500000000000003e-06 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:46103 5.7 5.949abb7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.461096+0000", 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9993529, 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000156021, 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 46103 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.461096+0000", 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.461096+0000", 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.461097+0000", 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3120000000000001e-06 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.461202+0000", 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104774 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.461212+0000", 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0144e-05 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.461221+0000", 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3829999999999992e-06 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:46082 5.0 e29/19 5:05acfbb3:::benchmark_data_smithi032_22800_object46081:head v 29'2951, mlcod=29'2948)", 2024-04-18T22:23:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:14.453707+0000", 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0067415879999999997, 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0075921859999999999, 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453707+0000", 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453707+0000", 2024-04-18T22:23:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453708+0000", 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0299999999999999e-07 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453729+0000", 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1316000000000001e-05 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453735+0000", 2024-04-18T22:23:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5040000000000002e-06 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453739+0000", 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1200000000000004e-06 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453783+0000", 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4620000000000003e-05 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:14.453794+0000", 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1015e-05 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 15 2024-04-18T22:23:14.523 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:14.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:14.525 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:14.522+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:14.572 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:23:14.572 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:14.572 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-04-18T22:23:14.572 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-18T22:23:14.572 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-18T22:23:14.573 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-04-18T22:23:14.625 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:14.622+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:14.682 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:14.682 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:14.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:14.715 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:14.715 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-18T22:23:14.716 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:14.726 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:14.723+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:14.826 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:14.827+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:14.856 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739796710999997, 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739664218999998, 2024-04-18T22:23:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739592633000001, 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739464380000001, 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739355052999997, 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739098716000001, 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.739011230000003, 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 42.738933244000002, 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0566461619999998, 2024-04-18T22:23:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0549897829999999, 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0548650659999996, 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0530854079999994, 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0526027370000008, 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0523636619999994, 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0522428910000006, 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0493534790000005, 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0492316600000002, 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0491212109999992, 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0490517429999997, 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.0489946020000005, 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:14.985 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:14.987 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:14.982+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:14.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:15.088 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:15.086+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:15.150 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:47599 5.8 5:17a910b8:::benchmark_data_smithi032_22800_object47598:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:15.149559+0000", 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0021719830000000002, 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022288049999999999, 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47599 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149559+0000", 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149559+0000", 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149560+0000", 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2800000000000005e-07 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149597+0000", 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7630999999999999e-05 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149610+0000", 2024-04-18T22:23:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2612999999999999e-05 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149615+0000", 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2820000000000004e-06 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149655+0000", 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0204000000000003e-05 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149740+0000", 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4474999999999993e-05 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149815+0000", 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5564999999999999e-05 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151514+0000", 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016989220000000001 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:47602 5.f 5:f0dfe747:::benchmark_data_smithi032_22800_object47601:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:15.151142+0000", 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00058839799999999996, 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085911500000000003, 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47602 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151142+0000", 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151142+0000", 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151143+0000", 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7700000000000004e-07 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151207+0000", 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4394999999999996e-05 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151221+0000", 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3603e-05 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151228+0000", 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8719999999999996e-06 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151274+0000", 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6153999999999999e-05 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151446+0000", 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017160499999999999 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151659+0000", 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021341399999999999 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:47603 5.9 5:96e9129e:::benchmark_data_smithi032_22800_object47602:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:15.151835+0000", 2024-04-18T22:23:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9998956, 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024215700000000001, 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47603 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151835+0000", 2024-04-18T22:23:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151835+0000", 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151836+0000", 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.06e-07 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151926+0000", 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0142000000000005e-05 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151934+0000", 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0830000000000004e-06 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151938+0000", 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4290000000000003e-06 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.151985+0000", 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6873999999999999e-05 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.152080+0000", 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5247000000000005e-05 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:47589 5.9 5:988c116f:::benchmark_data_smithi032_22800_object47588:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:15.145744+0000", 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0059862209999999999, 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064011550000000004, 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47589 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145744+0000", 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145744+0000", 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145745+0000", 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0100000000000004e-07 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145871+0000", 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125883 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145878+0000", 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-06 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145882+0000", 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.145925+0000", 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3127000000000002e-05 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146010+0000", 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5320000000000003e-05 2024-04-18T22:23:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146091+0000", 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0297999999999996e-05 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.147928+0000", 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018372690000000001 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:47592 5.d 5:b710288a:::benchmark_data_smithi032_22800_object47591:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:15.146094+0000", 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0056363339999999998, 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061268909999999998, 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47592 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146094+0000", 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146094+0000", 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146095+0000", 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7100000000000001e-07 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146194+0000", 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9261999999999999e-05 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146210+0000", 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6220000000000001e-05 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146215+0000", 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8080000000000004e-06 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146256+0000", 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146349+0000", 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2765999999999999e-05 2024-04-18T22:23:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146428+0000", 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9312000000000002e-05 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.148306+0000", 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001877809 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:47594 5.e 5:72b6d71d:::benchmark_data_smithi032_22800_object47593:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:15.146322+0000", 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0054083129999999997, 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059957969999999998, 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47594 2024-04-18T22:23:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146322+0000", 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146322+0000", 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146323+0000", 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9100000000000003e-07 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146406+0000", 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2905999999999995e-05 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146415+0000", 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5459999999999997e-06 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.146421+0000", 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.147807+0000", 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013859600000000001 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.147918+0000", 2024-04-18T22:23:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011018699999999999 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.148009+0000", 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1534999999999997e-05 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:15.149256+0000", 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012466949999999999 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2024-04-18T22:23:15.180 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:15.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:15.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 19 16 47702 47686 156.845 138.188 0.00991669 0.00637072 2024-04-18T22:23:15.223 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:15.222+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:15.289 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:15.286+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:15.376 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:15.376 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:15.389 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:15.386+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:15.394 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:15.491 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:15.486+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:15.590 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.476057803000003, 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:23:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:23:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:23:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473723911, 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:23:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473621012000002, 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:23:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:23:15.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:23:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473571344, 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:23:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473505668999998, 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:23:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473445922000003, 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:23:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:23:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473365852000001, 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:23:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:23:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473304444, 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:23:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.473220130999998, 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:23:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.466064398, 2024-04-18T22:23:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:23:15.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25664 5.f 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7875620320000003, 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093405780999999993, 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25664 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802645+0000", 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013290900000000001 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802656+0000", 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-05 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802661+0000", 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6729999999999997e-06 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802711+0000", 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9329000000000002e-05 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802804+0000", 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2929000000000004e-05 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802894+0000", 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0346999999999994e-05 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.807400+0000", 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045058290000000003 2024-04-18T22:23:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895676+0000", 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088275705999999995 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895810+0000", 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134635 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895917+0000", 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107309 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25666 5.f 5:f7add380:::benchmark_data_smithi032_22800_object25665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.787388515, 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093393754999999995, 2024-04-18T22:23:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25666 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802686+0000", 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802716+0000", 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9954000000000002e-05 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802723+0000", 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5689999999999999e-06 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802728+0000", 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6079999999999998e-06 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803716+0000", 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098778600000000005 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803871+0000", 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015491100000000001 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.804071+0000", 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020055600000000001 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.810020+0000", 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059493339999999997 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895982+0000", 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085962083999999994 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896036+0000", 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3813999999999997e-05 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896079+0000", 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2676000000000003e-05 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25673 5.d 5:becd3ab4:::benchmark_data_smithi032_22800_object25672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7820209990000002, 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085178675999999995, 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25673 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808054+0000", 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2900000000000002e-07 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808114+0000", 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9993e-05 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808126+0000", 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808136+0000", 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.064e-05 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808186+0000", 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9471000000000001e-05 2024-04-18T22:23:15.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808277+0000", 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0906e-05 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808387+0000", 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011026099999999999 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.812706+0000", 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043185289999999998 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.892987+0000", 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080281292000000004 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893126+0000", 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013872099999999999 2024-04-18T22:23:15.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893231+0000", 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010582300000000001 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25675 5.f 5:f60cbd32:::benchmark_data_smithi032_22800_object25674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7745794279999991, 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082348202999999995, 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25675 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815495+0000", 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815546+0000", 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0933999999999997e-05 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815572+0000", 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5936999999999999e-05 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815585+0000", 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2860000000000001e-05 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815664+0000", 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9334999999999999e-05 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815838+0000", 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017410700000000001 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.816021+0000", 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018247599999999999 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.817763+0000", 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001742494 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897556+0000", 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079792478999999999 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897717+0000", 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161199 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897843+0000", 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125415 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25682 5.b 5:d19b5078:::benchmark_data_smithi032_22800_object25681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6962620669999993, 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017001512999999999, 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25682 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4899999999999995e-07 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893846+0000", 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3546e-05 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893855+0000", 2024-04-18T22:23:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5689999999999994e-06 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893862+0000", 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7070000000000003e-06 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898400+0000", 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045379089999999997 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898589+0000", 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188965 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898831+0000", 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000242181 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902115+0000", 2024-04-18T22:23:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032839369999999998 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910719+0000", 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086036350000000001 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910771+0000", 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2584999999999999e-05 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910813+0000", 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1829999999999998e-05 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25693 5.e 5:715faa37:::benchmark_data_smithi032_22800_object25692:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6908538689999997, 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016385088999999999, 2024-04-18T22:23:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25693 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899222+0000", 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.646e-06 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899274+0000", 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2642999999999997e-05 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899292+0000", 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8196e-05 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899304+0000", 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1866e-05 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899425+0000", 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121131 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899751+0000", 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032516000000000001 2024-04-18T22:23:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.900183+0000", 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043271999999999999 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902912+0000", 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027285930000000001 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915375+0000", 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012463093 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915503+0000", 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012793099999999999 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915605+0000", 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010211 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26099 5.f 5:fd0a425b:::benchmark_data_smithi032_22800_object26098:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4977484640000007, 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01621906, 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26099 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092326+0000", 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.096e-06 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092379+0000", 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2737e-05 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092394+0000", 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4545999999999999e-05 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092399+0000", 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7320000000000003e-06 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.099921+0000", 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075215849999999999 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100044+0000", 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122786 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100132+0000", 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8230000000000001e-05 2024-04-18T22:23:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102949+0000", 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028173769999999998 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108443+0000", 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005493698 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108504+0000", 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1211999999999996e-05 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108544+0000", 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0061000000000002e-05 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26100 5.f 5:fb6fac53:::benchmark_data_smithi032_22800_object26099:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4975705670000004, 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016127091999999999, 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26100 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092504+0000", 2024-04-18T22:23:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0299999999999997e-07 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092632+0000", 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012751599999999999 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092644+0000", 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2618e-05 2024-04-18T22:23:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092653+0000", 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100929+0000", 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082759610000000001 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101092+0000", 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016214300000000001 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101249+0000", 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015756099999999999 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.104207+0000", 2024-04-18T22:23:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002957758 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108553+0000", 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043464280000000003 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108596+0000", 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3135e-05 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108630+0000", 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3798999999999999e-05 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26931 5.b 5:d9657119:::benchmark_data_smithi032_22800_object26930:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1701075529999994, 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016342974999999999, 2024-04-18T22:23:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26931 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419967+0000", 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420067+0000", 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9531999999999995e-05 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420075+0000", 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420082+0000", 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-04-18T22:23:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421326+0000", 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012431720000000001 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421410+0000", 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4254999999999999e-05 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421491+0000", 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0814000000000002e-05 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.422721+0000", 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012302109999999999 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436212+0000", 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013491385999999999 2024-04-18T22:23:15.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436268+0000", 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5585999999999999e-05 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436309+0000", 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1374000000000001e-05 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:35577 5.f 5:f7bab6df:::benchmark_data_smithi032_22800_object35576:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:09.969017+0000", 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6210567979999997, 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015645526, 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35577 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969017+0000", 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969017+0000", 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:15.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969017+0000", 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5899999999999996e-07 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969046+0000", 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8733000000000001e-05 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969053+0000", 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6019999999999996e-06 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969057+0000", 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1810000000000003e-06 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969103+0000", 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6428999999999999e-05 2024-04-18T22:23:15.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969241+0000", 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137508 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.969389+0000", 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014769100000000001 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.970880+0000", 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001491561 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.984600+0000", 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013719543000000001 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.984631+0000", 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1013000000000003e-05 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:09.984662+0000", 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1705999999999998e-05 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:15.719 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:15.721 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:15.722 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:15.719+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:15.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:15.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:15.819+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:49266 5.c e29/19 5:3eababcd:::benchmark_data_smithi032_22800_object49265:head v 29'3069, mlcod=29'3067)", 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:15.874274+0000", 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001065554, 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001087099, 2024-04-18T22:23:15.874 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874274+0000", 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874274+0000", 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874275+0000", 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4499999999999997e-07 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874341+0000", 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6780999999999996e-05 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874349+0000", 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9659999999999996e-06 2024-04-18T22:23:15.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874354+0000", 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3549999999999998e-06 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874403+0000", 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9731000000000002e-05 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874425+0000", 2024-04-18T22:23:15.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1583999999999999e-05 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:49269 5.2 e29/23 5:4c61debe:::benchmark_data_smithi032_22800_object49268:head v 29'3075, mlcod=29'3073)", 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:15.874369+0000", 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00097077199999999995, 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001053544, 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:15.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874369+0000", 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874369+0000", 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874369+0000", 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3099999999999999e-07 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874434+0000", 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5103000000000003e-05 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874439+0000", 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1710000000000001e-06 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874443+0000", 2024-04-18T22:23:15.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4039999999999999e-06 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874480+0000", 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6936e-05 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874492+0000", 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2028000000000001e-05 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:49271 5.d e29/19 5:b33e005b:::benchmark_data_smithi032_22800_object49270:head v 29'3121, mlcod=29'3118)", 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:15.874864+0000", 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00047532899999999998, 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00061386100000000005, 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874864+0000", 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874864+0000", 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874865+0000", 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4900000000000001e-07 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874959+0000", 2024-04-18T22:23:15.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4073999999999994e-05 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874966+0000", 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0999999999999998e-06 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.874970+0000", 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8809999999999998e-06 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875016+0000", 2024-04-18T22:23:15.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6192000000000001e-05 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875029+0000", 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3275000000000001e-05 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:49273 5.9 e29/23 5:9f689e76:::benchmark_data_smithi032_22800_object49272:head v 29'3059, mlcod=29'3057)", 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:15.875281+0000", 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 5.8799999999999999e-05, 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025107100000000001, 2024-04-18T22:23:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875281+0000", 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875281+0000", 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875281+0000", 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875331+0000", 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9311999999999998e-05 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875342+0000", 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0882e-05 2024-04-18T22:23:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875347+0000", 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6380000000000001e-06 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875401+0000", 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3724000000000003e-05 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:15.875415+0000", 2024-04-18T22:23:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3967e-05 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 4 2024-04-18T22:23:15.889 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:15.892 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:15.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:23:15.921 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:15.921 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:15.922 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:15.918+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:15.939 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:15.957 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:16.024 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:16.023+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:16.081 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:16.082 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:16.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:16.124 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:16.122+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:16.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:2024-04-18T22:23:16.193193+0000 min lat: 0.00133878 max lat: 1.02957 avg lat: 0.00639514 2024-04-18T22:23:16.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-18T22:23:16.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 20 16 50022 50006 156.252 145 0.00737326 0.00639514 2024-04-18T22:23:16.225 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:16.222+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:16.290 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:16.291 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:16.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:16.357 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:16.359+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:16.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:16.427+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:16.465 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:16.465 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:16.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:16.533 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:16.535+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:16.628 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:16.627+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:16.690 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:16.690 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:16.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:16.753 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:16.755+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:16.830 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:16.831+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:23:16.906 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:16.906 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:16.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:16.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:16.931+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:17.031 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:17.030+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:17.081 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:17.081 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:17.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:17.131 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:17.131+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:17.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 21 16 52126 52110 155.073 131.5 0.00988679 0.00644297 2024-04-18T22:23:17.232 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:17.231+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:17.296 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:17.296 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:17.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:17.333 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:17.335+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:17.434 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:17.431+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:17.466 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:17.466 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:17.488 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:17.535 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:17.535+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:17.635 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:17.635 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:17.636 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:17.635+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:17.657 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:17.737 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:17.735+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:17.838 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:17.839+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:17.840 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:17.840 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:17.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:17.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:17.939+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:17.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:18.004 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:18.004 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:18.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:18.070 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:18.071+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:18.140 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:18.138+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:18.147 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:18.147 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:18.166 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54218 5.7 5:e1cf7419:::benchmark_data_smithi032_22800_object54217:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.173843+0000", 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0058304120000000001, 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0058572909999999997, 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.179 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54218 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173843+0000", 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173843+0000", 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173844+0000", 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8000000000000004e-07 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173946+0000", 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102138 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173957+0000", 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.131e-05 2024-04-18T22:23:18.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173966+0000", 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6000000000000007e-06 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.175037+0000", 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001070987 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.175162+0000", 2024-04-18T22:23:18.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012500400000000001 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.175307+0000", 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014533600000000001 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179444+0000", 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041365530000000003 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179680+0000", 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000235882 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54216 5.b e30/19 5:d5379316:::benchmark_data_smithi032_22800_object54215:head v 30'3424, mlcod=30'3418)", 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.173846+0000", 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0058274529999999998, 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005941755, 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173846+0000", 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173846+0000", 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173847+0000", 2024-04-18T22:23:18.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173973+0000", 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000126506 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173984+0000", 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0477e-05 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173990+0000", 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8900000000000001e-06 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176205+0000", 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022142720000000002 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176222+0000", 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7411e-05 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.187 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54220 5.7 5:e490f09c:::benchmark_data_smithi032_22800_object54219:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.173983+0000", 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0056899999999999997, 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0058677649999999996, 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54220 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173983+0000", 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173983+0000", 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.173984+0000", 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8999999999999995e-07 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.174023+0000", 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8639000000000001e-05 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.174032+0000", 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.166e-06 2024-04-18T22:23:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.174038+0000", 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0399999999999998e-06 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176917+0000", 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028785360000000001 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177031+0000", 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114212 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177179+0000", 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014829200000000001 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179453+0000", 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002274238 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.192 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54224 5.c 5:3207e8a2:::benchmark_data_smithi032_22800_object54223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.176611+0000", 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030624409999999999, 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033165170000000002, 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54224 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176611+0000", 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176611+0000", 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176612+0000", 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4200000000000005e-07 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176680+0000", 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8303e-05 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176698+0000", 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8309999999999999e-05 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176707+0000", 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1459999999999996e-06 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176759+0000", 2024-04-18T22:23:18.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2120000000000002e-05 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176841+0000", 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2656000000000003e-05 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176916+0000", 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4308999999999996e-05 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179478+0000", 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002562412 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54223 5.4 e30/19 5:213d8c73:::benchmark_data_smithi032_22800_object54222:head v 30'3469, mlcod=30'3467)", 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.176992+0000", 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026813060000000001, 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030054389999999999, 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176992+0000", 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176992+0000", 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176993+0000", 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5499999999999998e-07 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177055+0000", 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2339000000000001e-05 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177064+0000", 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1449999999999997e-06 2024-04-18T22:23:18.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177070+0000", 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3300000000000004e-06 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177116+0000", 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5827e-05 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177132+0000", 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5931e-05 2024-04-18T22:23:18.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54225 5.8 e30/19 5:1c30a514:::benchmark_data_smithi032_22800_object54224:head v 30'3335, mlcod=30'3333)", 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.177279+0000", 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023940020000000001, 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027724350000000002, 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177279+0000", 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177279+0000", 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177280+0000", 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2900000000000004e-07 2024-04-18T22:23:18.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177336+0000", 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6008999999999999e-05 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177345+0000", 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8030000000000003e-06 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177351+0000", 2024-04-18T22:23:18.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1e-06 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177396+0000", 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5325999999999999e-05 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177412+0000", 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.605e-05 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54226 5.b e30/19 5:d0512203:::benchmark_data_smithi032_22800_object54225:head v 30'3425, mlcod=30'3420)", 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.177374+0000", 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022989999999999998, 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027434930000000001, 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177374+0000", 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177374+0000", 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177375+0000", 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7300000000000002e-07 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177424+0000", 2024-04-18T22:23:18.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8738e-05 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177431+0000", 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1389999999999998e-06 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177436+0000", 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9439999999999996e-06 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179135+0000", 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001699248 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179145+0000", 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6730000000000004e-06 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54227 5.0 e30/19 5:053ba362:::benchmark_data_smithi032_22800_object54226:head v 30'3465, mlcod=30'3462)", 2024-04-18T22:23:18.209 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.177762+0000", 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001910964, 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024216929999999999, 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177762+0000", 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177762+0000", 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177763+0000", 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0500000000000004e-07 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177825+0000", 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2343999999999998e-05 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177832+0000", 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1840000000000002e-06 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177838+0000", 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1050000000000003e-06 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177890+0000", 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2598e-05 2024-04-18T22:23:18.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.177907+0000", 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6983999999999999e-05 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54229 5.a 5:5dd67256:::benchmark_data_smithi032_22800_object54228:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.179735+0000", 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999385, 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00052877999999999996, 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54229 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179735+0000", 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179735+0000", 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179736+0000", 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3099999999999996e-07 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179840+0000", 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104177 2024-04-18T22:23:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179849+0000", 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4789999999999996e-06 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179853+0000", 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7160000000000002e-06 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179890+0000", 2024-04-18T22:23:18.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6359999999999997e-05 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179981+0000", 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1070999999999996e-05 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180090+0000", 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109158 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54232 5.2 5:44054b7f:::benchmark_data_smithi032_22800_object54231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.179892+0000", 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997811, 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046186399999999998, 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54232 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179892+0000", 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179892+0000", 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179893+0000", 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0100000000000004e-07 2024-04-18T22:23:18.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179955+0000", 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2495000000000004e-05 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179963+0000", 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.909e-06 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179968+0000", 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180010+0000", 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1752999999999998e-05 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180140+0000", 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013008299999999999 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180274+0000", 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000134678 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54233 5.6 5:6a2a562a:::benchmark_data_smithi032_22800_object54232:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.179977+0000", 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9996963, 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00049305599999999996, 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54233 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179977+0000", 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179977+0000", 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.179978+0000", 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8100000000000003e-07 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180009+0000", 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1622000000000003e-05 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180014+0000", 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5199999999999999e-06 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180017+0000", 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3299999999999999e-06 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180070+0000", 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3035999999999998e-05 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180191+0000", 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120698 2024-04-18T22:23:18.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180303+0000", 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112308 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54228 5.4 e30/19 5:24fede8a:::benchmark_data_smithi032_22800_object54227:head v 30'3470, mlcod=30'3467)", 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.180130+0000", 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9995432, 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000406103, 2024-04-18T22:23:18.226 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180130+0000", 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180130+0000", 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180131+0000", 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3200000000000005e-07 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180193+0000", 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2415999999999993e-05 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180199+0000", 2024-04-18T22:23:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3049999999999998e-06 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180203+0000", 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1080000000000003e-06 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180242+0000", 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8222000000000001e-05 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180311+0000", 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8895999999999999e-05 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54234 5.5 5:afef9d26:::benchmark_data_smithi032_22800_object54233:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.180252+0000", 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9994216, 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00034189099999999998, 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54234 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180252+0000", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180252+0000", 2024-04-18T22:23:18.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180253+0000", 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7199999999999998e-07 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180319+0000", 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6359000000000006e-05 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180327+0000", 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0749999999999998e-06 2024-04-18T22:23:18.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180333+0000", 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.195e-06 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180384+0000", 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0986999999999998e-05 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180485+0000", 2024-04-18T22:23:18.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100519 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.180591+0000", 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106352 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:54211 5.b e30/19 5:de0b9562:::benchmark_data_smithi032_22800_object54210:head v 30'3423, mlcod=30'3418)", 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.171534+0000", 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0081394610000000006, 2024-04-18T22:23:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091291250000000001, 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171534+0000", 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171534+0000", 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171535+0000", 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3900000000000004e-07 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171596+0000", 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1152e-05 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171607+0000", 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1627999999999999e-05 2024-04-18T22:23:18.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171614+0000", 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9770000000000003e-06 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.172875+0000", 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001261007 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.172902+0000", 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6506e-05 2024-04-18T22:23:18.237 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:54213 5.3 5:c716112a:::benchmark_data_smithi032_22800_object54212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:18.171812+0000", 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0078616610000000007, 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0089146109999999994, 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 54213 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171812+0000", 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171812+0000", 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171812+0000", 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1099999999999995e-07 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171863+0000", 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0943999999999999e-05 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171872+0000", 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6829999999999997e-06 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.171877+0000", 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.04e-06 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176189+0000", 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043118089999999998 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176353+0000", 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000164473 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:18.176513+0000", 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015954700000000001 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.242 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:18.243 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 15 2024-04-18T22:23:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:18.244 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:18.244 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:18.244 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 22 16 54259 54243 154.083 133.312 0.00810721 0.00648484 2024-04-18T22:23:18.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:18.246 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:18.243+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:18.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:18.343+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:18.442 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:18.442 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:18.442 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:18.442 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:18.443 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:18.446 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:18.442+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:18.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:18.547 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:18.547+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:18.611 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495927709999997, 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.612 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:18.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:18.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:18.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495795217999998, 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:18.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.624 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495723632000001, 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:18.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.630 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495595379000001, 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:18.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495486051999997, 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:18.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:18.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:18.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495229715000001, 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:18.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:18.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.648 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495142229000002, 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:18.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:18.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:18.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 46.495064243000002, 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.812777161, 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:18.658 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.811120782, 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:18.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:18.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.810996064999999, 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.670 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:18.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.675 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.809216406999999, 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.681 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.808733736000001, 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.808494660999999, 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:18.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:18.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.80837389, 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:18.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:18.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.805484478, 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.805362659, 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.805252210000001, 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.710 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:18.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.805182742, 2024-04-18T22:23:18.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:18.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:18.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.805125601, 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:18.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:18.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:18.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:18.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:18.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:18.727 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:18.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:18.729 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:18.726+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:18.829 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:18.831+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:18.884 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4586.0:55709 5.4 e30/19 ondisk, result = 0)", 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:18.885296+0000", 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.8362000000000005e-05, 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112495, 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885296+0000", 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885296+0000", 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885296+0000", 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885302+0000", 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6400000000000002e-06 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885313+0000", 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0733999999999999e-05 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885318+0000", 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0989999999999998e-06 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885358+0000", 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0311000000000001e-05 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885363+0000", 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4586.0:55711 5.4 e30/19 ondisk, result = 0)", 2024-04-18T22:23:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:18.885342+0000", 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.2444999999999998e-05, 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014124499999999999, 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885342+0000", 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885342+0000", 2024-04-18T22:23:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885342+0000", 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1199999999999999e-07 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885344+0000", 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885352+0000", 2024-04-18T22:23:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885357+0000", 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885467+0000", 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011069900000000001 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885472+0000", 2024-04-18T22:23:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0629999999999999e-06 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:55711 5.4 5:26d76adb:::benchmark_data_smithi032_22800_object55710:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:18.878703+0000", 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0066814800000000001, 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068463090000000001, 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:18.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 55711 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.878703+0000", 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.878703+0000", 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.878703+0000", 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.878792+0000", 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8799000000000001e-05 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.878804+0000", 2024-04-18T22:23:18.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1552999999999999e-05 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.878811+0000", 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6490000000000008e-06 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880307+0000", 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014951260000000001 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:18.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880448+0000", 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141125 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880569+0000", 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120888 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.883798+0000", 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032297710000000002 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:18.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885477+0000", 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016782489999999999 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885518+0000", 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1087999999999997e-05 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885549+0000", 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1149999999999998e-05 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:55713 5.4 5:2e12233a:::benchmark_data_smithi032_22800_object55712:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:18.880516+0000", 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0048681369999999998, 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051680770000000001, 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 55713 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880516+0000", 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880516+0000", 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880517+0000", 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880599+0000", 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1736999999999993e-05 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880609+0000", 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0586e-05 2024-04-18T22:23:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880616+0000", 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.881966+0000", 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001349685 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.882129+0000", 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162566 2024-04-18T22:23:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.882297+0000", 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016802599999999999 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:55715 5.8 5:1f8e3678:::benchmark_data_smithi032_22800_object55714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:18.880694+0000", 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0046900759999999996, 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050362150000000001, 2024-04-18T22:23:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 55715 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880694+0000", 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880694+0000", 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880695+0000", 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8700000000000005e-07 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880775+0000", 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.975e-05 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880792+0000", 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7003e-05 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880799+0000", 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.909e-06 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880837+0000", 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7215999999999997e-05 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.880944+0000", 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010696699999999999 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.881031+0000", 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7820999999999998e-05 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884885+0000", 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038533920000000002 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:55721 5.f 5:f44c0e0f:::benchmark_data_smithi032_22800_object55720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:18.884908+0000", 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000476164, 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000910303, 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 55721 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884908+0000", 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884908+0000", 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884909+0000", 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1960000000000001e-06 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884970+0000", 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0881000000000002e-05 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884984+0000", 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4293e-05 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.884994+0000", 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2920000000000001e-06 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885047+0000", 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3081999999999998e-05 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885136+0000", 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9066000000000004e-05 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:18.885217+0000", 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1520000000000006e-05 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2024-04-18T22:23:18.911 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:18.912 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:18.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:23:18.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:18.927+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:19.031 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:19.031+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:19.075 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:19.076 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:19.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:19.139 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:19.138+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:19.191 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 23 16 56370 56354 153.119 131.938 0.00716828 0.00652562 2024-04-18T22:23:19.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:19.231+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:19.253 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:19.253 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:23:19.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:23:19.253 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:19.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.138514897, 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:23:19.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:23:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.136181004999997, 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:23:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:23:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.136078105999999, 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:23:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:23:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.136028437999997, 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:23:19.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:23:19.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:23:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.135962763000002, 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:23:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:23:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:23:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.135903016, 2024-04-18T22:23:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:23:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.135822945999998, 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:23:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:23:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.135761537999997, 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:23:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.135677225000002, 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:23:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:23:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.128521491999997, 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:23:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:23:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25664 5.f 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.450019126000001, 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093405780999999993, 2024-04-18T22:23:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25664 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802645+0000", 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013290900000000001 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802656+0000", 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-05 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802661+0000", 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6729999999999997e-06 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802711+0000", 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9329000000000002e-05 2024-04-18T22:23:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802804+0000", 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2929000000000004e-05 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802894+0000", 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0346999999999994e-05 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.807400+0000", 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045058290000000003 2024-04-18T22:23:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895676+0000", 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088275705999999995 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895810+0000", 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134635 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895917+0000", 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107309 2024-04-18T22:23:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25666 5.f 5:f7add380:::benchmark_data_smithi032_22800_object25665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.449845609, 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093393754999999995, 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25666 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802686+0000", 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802716+0000", 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9954000000000002e-05 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802723+0000", 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5689999999999999e-06 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802728+0000", 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6079999999999998e-06 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803716+0000", 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098778600000000005 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803871+0000", 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015491100000000001 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.804071+0000", 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020055600000000001 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.810020+0000", 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059493339999999997 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895982+0000", 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085962083999999994 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896036+0000", 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3813999999999997e-05 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896079+0000", 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2676000000000003e-05 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25673 5.d 5:becd3ab4:::benchmark_data_smithi032_22800_object25672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.444478093000001, 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085178675999999995, 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25673 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808054+0000", 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2900000000000002e-07 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808114+0000", 2024-04-18T22:23:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9993e-05 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808126+0000", 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808136+0000", 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.064e-05 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808186+0000", 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9471000000000001e-05 2024-04-18T22:23:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808277+0000", 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0906e-05 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808387+0000", 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011026099999999999 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.812706+0000", 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043185289999999998 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.892987+0000", 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080281292000000004 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893126+0000", 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013872099999999999 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893231+0000", 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010582300000000001 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25675 5.f 5:f60cbd32:::benchmark_data_smithi032_22800_object25674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.437036522, 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082348202999999995, 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25675 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815495+0000", 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815546+0000", 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0933999999999997e-05 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815572+0000", 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5936999999999999e-05 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815585+0000", 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2860000000000001e-05 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815664+0000", 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9334999999999999e-05 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815838+0000", 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017410700000000001 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.816021+0000", 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018247599999999999 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.817763+0000", 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001742494 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897556+0000", 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079792478999999999 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897717+0000", 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161199 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897843+0000", 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125415 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25682 5.b 5:d19b5078:::benchmark_data_smithi032_22800_object25681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.358719161, 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017001512999999999, 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25682 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4899999999999995e-07 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893846+0000", 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3546e-05 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893855+0000", 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5689999999999994e-06 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893862+0000", 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7070000000000003e-06 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898400+0000", 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045379089999999997 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898589+0000", 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188965 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898831+0000", 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000242181 2024-04-18T22:23:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902115+0000", 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032839369999999998 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910719+0000", 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086036350000000001 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910771+0000", 2024-04-18T22:23:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2584999999999999e-05 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910813+0000", 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1829999999999998e-05 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25693 5.e 5:715faa37:::benchmark_data_smithi032_22800_object25692:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.353310963, 2024-04-18T22:23:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016385088999999999, 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25693 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899220+0000", 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899222+0000", 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.646e-06 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899274+0000", 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2642999999999997e-05 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899292+0000", 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8196e-05 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899304+0000", 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1866e-05 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899425+0000", 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121131 2024-04-18T22:23:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.899751+0000", 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032516000000000001 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.900183+0000", 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043271999999999999 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902912+0000", 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027285930000000001 2024-04-18T22:23:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915375+0000", 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012463093 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915503+0000", 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012793099999999999 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.915605+0000", 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010211 2024-04-18T22:23:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26099 5.f 5:fd0a425b:::benchmark_data_smithi032_22800_object26098:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.160205557999999, 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01621906, 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26099 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092325+0000", 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092326+0000", 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.096e-06 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092379+0000", 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2737e-05 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092394+0000", 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4545999999999999e-05 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092399+0000", 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7320000000000003e-06 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.099921+0000", 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075215849999999999 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100044+0000", 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122786 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100132+0000", 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8230000000000001e-05 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.102949+0000", 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028173769999999998 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108443+0000", 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005493698 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108504+0000", 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1211999999999996e-05 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108544+0000", 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0061000000000002e-05 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26100 5.f 5:fb6fac53:::benchmark_data_smithi032_22800_object26099:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.160027661000001, 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016127091999999999, 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26100 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092503+0000", 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092504+0000", 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0299999999999997e-07 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092632+0000", 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012751599999999999 2024-04-18T22:23:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092644+0000", 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2618e-05 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.092653+0000", 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.100929+0000", 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082759610000000001 2024-04-18T22:23:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101092+0000", 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016214300000000001 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.101249+0000", 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015756099999999999 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.104207+0000", 2024-04-18T22:23:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002957758 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108553+0000", 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043464280000000003 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108596+0000", 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3135e-05 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.108630+0000", 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3798999999999999e-05 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:26931 5.b 5:d9657119:::benchmark_data_smithi032_22800_object26930:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.832564647, 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016342974999999999, 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26931 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419966+0000", 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.419967+0000", 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420067+0000", 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9531999999999995e-05 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420075+0000", 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-04-18T22:23:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.420082+0000", 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421326+0000", 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012431720000000001 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421410+0000", 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4254999999999999e-05 2024-04-18T22:23:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.421491+0000", 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0814000000000002e-05 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.422721+0000", 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012302109999999999 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436212+0000", 2024-04-18T22:23:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013491385999999999 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436268+0000", 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5585999999999999e-05 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:06.436309+0000", 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1374000000000001e-05 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:56088 5.0 5:0839553e:::benchmark_data_smithi032_22800_object56087:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:19.055580+0000", 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.19695052399999999, 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017740243999999999, 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 56088 2024-04-18T22:23:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055580+0000", 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055580+0000", 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055581+0000", 2024-04-18T22:23:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0800000000000003e-07 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055721+0000", 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013982999999999999 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055730+0000", 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3340000000000008e-06 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055748+0000", 2024-04-18T22:23:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7776000000000001e-05 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.064792+0000", 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090440720000000002 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.064933+0000", 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014087099999999999 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.065100+0000", 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016689 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.069278+0000", 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004177648 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.073223+0000", 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039457870000000001 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.073277+0000", 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3899e-05 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.073321+0000", 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3229000000000002e-05 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:19.363 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:19.365 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:19.365 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:19.362+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:19.465 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:19.462+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:57053 5.9 e30/23 5:9d62125d:::benchmark_data_smithi032_22800_object57052:head v 30'3522, mlcod=30'3519)", 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:19.510437+0000", 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002977305, 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029938590000000002, 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.512 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510437+0000", 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510437+0000", 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510438+0000", 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.89e-07 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510582+0000", 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014449299999999999 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510595+0000", 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3148e-05 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510603+0000", 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1969999999999996e-06 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510651+0000", 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7481999999999997e-05 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.510662+0000", 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1131999999999999e-05 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:57052 5.7 e30/23 5:e5ad1047:::benchmark_data_smithi032_22800_object57051:head v 30'3581, mlcod=30'3576)", 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:19.511040+0000", 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0023744270000000001, 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024282940000000001, 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511040+0000", 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511040+0000", 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511041+0000", 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9299999999999998e-07 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511094+0000", 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3242999999999998e-05 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511105+0000", 2024-04-18T22:23:19.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0986e-05 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511112+0000", 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2219999999999996e-06 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511394+0000", 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028201499999999999 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511410+0000", 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.624e-05 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:57057 5.c e30/19 5:395be6f6:::benchmark_data_smithi032_22800_object57056:head v 30'3572, mlcod=30'3570)", 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:19.511655+0000", 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0017590360000000001, 2024-04-18T22:23:19.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001856147, 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511655+0000", 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511655+0000", 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511656+0000", 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2600000000000002e-07 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511707+0000", 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1440000000000002e-05 2024-04-18T22:23:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511717+0000", 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9769999999999995e-06 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511724+0000", 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5919999999999997e-06 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511765+0000", 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0742999999999999e-05 2024-04-18T22:23:19.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:19.511777+0000", 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2055000000000001e-05 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 3 2024-04-18T22:23:19.524 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:19.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:19.566 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:19.566+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:19.667 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:19.666+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:19.671 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:19.671 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:19.671 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:19.671 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:19.671 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:19.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:19.737 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:23:19.737 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:19.737 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-18T22:23:19.737 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-04-18T22:23:19.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-18T22:23:19.767 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:19.766+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.834 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718624269999999, 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:19.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:19.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:19.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:19.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:19.841 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.841 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.841 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718555672000001, 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:19.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:19.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:19.847 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718459826, 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:19.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:19.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:19.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718402943999997, 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:19.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:19.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:19.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718343271999998, 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.859 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:19.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718285309999999, 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.865 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.870 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718065617999997, 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:19.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:19.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.718008689000001, 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.876 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:19.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:19.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.717912802999997, 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:19.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.887 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.717850474999999, 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:19.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:19.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:19.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.717777996999999, 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.717625529000003, 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:19.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:19.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:19.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.717546806000001, 2024-04-18T22:23:19.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:19.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:19.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:19.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:19.909 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.716025950000002, 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:19.915 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.715927598, 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:19.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.034205645, 2024-04-18T22:23:19.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:19.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.923 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.033551431999999, 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.927 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.033444737, 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:19.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.031578828000001, 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.932 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:19.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:19.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.030913504999999, 2024-04-18T22:23:19.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:19.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:19.940 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:19.942 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:19.938+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:23:19.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:20.043 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:20.042+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:20.100 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:20.100 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:20.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.100 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:58266 5.1 5:880730eb:::benchmark_data_smithi032_22800_object58265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-04-18T22:23:20.100 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:20.100431+0000", 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0016734300000000001, 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016993100000000001, 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 58266 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:20.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100431+0000", 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100431+0000", 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100431+0000", 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9299999999999997e-07 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100524+0000", 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2927e-05 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100539+0000", 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5155e-05 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100548+0000", 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0360000000000006e-06 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100591+0000", 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3310999999999999e-05 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100708+0000", 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116725 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:20.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100828+0000", 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120798 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:58264 5.0 e31/19 5:087b61c5:::benchmark_data_smithi032_22800_object58263:head v 31'3706, mlcod=31'3704)", 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001357322, 2024-04-18T22:23:20.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001456525, 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100803+0000", 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5402000000000003e-05 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100812+0000", 2024-04-18T22:23:20.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3749999999999992e-06 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100820+0000", 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1710000000000002e-06 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100871+0000", 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0875000000000003e-05 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100890+0000", 2024-04-18T22:23:20.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.853e-05 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:58267 5.5 5:ac9172e5:::benchmark_data_smithi032_22800_object58266:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001357362, 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001515244, 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:20.109 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 58267 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100747+0000", 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4700000000000001e-07 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:20.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100818+0000", 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0548000000000003e-05 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100826+0000", 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9689999999999994e-06 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100832+0000", 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2400000000000004e-06 2024-04-18T22:23:20.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.100894+0000", 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1419000000000003e-05 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101016+0000", 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121962 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:20.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101150+0000", 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013416900000000001 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:58265 5.b e31/19 5:d6deaf91:::benchmark_data_smithi032_22800_object58264:head v 31'3687, mlcod=31'3684)", 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:20.101504+0000", 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00060009400000000002, 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00082306100000000004, 2024-04-18T22:23:20.114 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:20.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:20.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:20.115 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:20.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:20.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101504+0000", 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101504+0000", 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101505+0000", 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:23:20.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101549+0000", 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4796000000000003e-05 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101563+0000", 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4039000000000001e-05 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101569+0000", 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0530000000000003e-06 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101616+0000", 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6131000000000003e-05 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:20.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101636+0000", 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0361e-05 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:58261 5.a e31/19)", 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:20.101999+0000", 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00010468599999999999, 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037843399999999998, 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:20.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101999+0000", 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101999+0000", 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.102000+0000", 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9900000000000001e-07 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.102005+0000", 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7840000000000003e-06 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.102014+0000", 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2890000000000003e-06 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.102019+0000", 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2750000000000003e-06 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:58261 5.a 5:52a6cee4:::benchmark_data_smithi032_22800_object58260:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-04-18T22:23:20.122 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:20.097898+0000", 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0042056109999999997, 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045276470000000001, 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 58261 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:20.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.097898+0000", 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.097898+0000", 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.097899+0000", 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8899999999999999e-07 2024-04-18T22:23:20.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.097952+0000", 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2904e-05 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.097961+0000", 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5799999999999992e-06 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.097979+0000", 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8312e-05 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.098028+0000", 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9326999999999997e-05 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.098160+0000", 2024-04-18T22:23:20.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013206900000000001 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.098315+0000", 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015460100000000001 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:20.101115+0000", 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002799658 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:20.128 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:20.128 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 6 2024-04-18T22:23:20.128 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:20.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:20.144 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:20.143+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:20.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 24 16 58453 58437 152.163 130.188 0.0101865 0.00656668 2024-04-18T22:23:20.245 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:20.242+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:20.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:20.332 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:20.332 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:20.332 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:20.332 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:20.332 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:20.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:20.347+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:20.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:20.407 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:20.408 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:20.427 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:20.428 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:20.447 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:20.442+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:20.448 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:20.448 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:20.548 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:20.546+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:20.551 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:20.551 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:20.572 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:20.649 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:20.646+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:20.725 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:20.725 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:20.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:20.791 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:20.790+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:20.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:20.851+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:20.898 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:20.898 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:20.915 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-18T22:23:20.920 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:20.951 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:20.950+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:21.052 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:21.050+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:21.119 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:21.119 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:21.140 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:21.152 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:21.150+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:21.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 25 16 60523 60507 151.251 129.375 0.00582626 0.00660706 2024-04-18T22:23:21.254 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:21.251+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:21.282 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:21.283 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:21.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:21.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:21.391+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:21.455 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:21.455+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:21.497 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:21.497 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:21.517 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:21.556 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:21.554+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:21.657 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:21.655+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:21.709 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:21.709 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:21.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:21.758 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:21.754+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:21.860 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:21.858+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:21.887 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:21.887 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:21.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:21.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:23:21.961 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:21.958+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:22.062 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:22.062+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:22.101 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:22.102 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:22.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:22.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:22.162+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:22.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 26 16 62558 62542 150.325 127.188 0.00869965 0.00664727 2024-04-18T22:23:22.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:22.263+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:22.309 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:22.309 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:22.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:22.371 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:22.370+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:22.466 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:22.466+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:22.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:22.479 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:22.479 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:22.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:22.566 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:22.566+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:22.635 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:22.653 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:22.653 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:22.667 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:22.666+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:22.671 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:22.698 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:22.698 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:22.698 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:22.698 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:22.698 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:22.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:22.768 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:22.766+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:22.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:22.867+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:22.881 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:13 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.767482143000002, 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028996119000000001, 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114016+0000", 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114019+0000", 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6630000000000002e-06 2024-04-18T22:23:22.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114052+0000", 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3671999999999999e-05 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114083+0000", 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0781999999999998e-05 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.114106+0000", 2024-04-18T22:23:22.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2748999999999999e-05 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122493+0000", 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083874759999999996 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122675+0000", 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182213 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122886+0000", 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000210261 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132645+0000", 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097590950000000006 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141469+0000", 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088246409999999994 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142553+0000", 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010831510000000001 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.143012+0000", 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045941599999999998 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:14 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.765148250999999, 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026000539, 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116350+0000", 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116352+0000", 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116367+0000", 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4927e-05 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116394+0000", 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7212e-05 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116420+0000", 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5989000000000001e-05 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122615+0000", 2024-04-18T22:23:22.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061949509999999998 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.122888+0000", 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272899 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123191+0000", 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030346799999999999 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.132627+0000", 2024-04-18T22:23:22.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094357380000000008 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.141990+0000", 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093627869999999992 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142219+0000", 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022891099999999999 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142350+0000", 2024-04-18T22:23:22.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131531 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:17 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.765045352000001, 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024303908999999999, 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116453+0000", 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116454+0000", 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.19e-06 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4130000000000003e-06 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116475+0000", 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4413000000000001e-05 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116484+0000", 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4700000000000008e-06 2024-04-18T22:23:22.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.125727+0000", 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092431279999999998 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126053+0000", 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032563999999999998 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.126460+0000", 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040692099999999998 2024-04-18T22:23:22.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133294+0000", 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00683367 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140493+0000", 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071990939999999996 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140648+0000", 2024-04-18T22:23:22.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015541099999999999 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140757+0000", 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108559 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:27 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.764995683999999, 2024-04-18T22:23:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038571828000000002, 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:22.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116503+0000", 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7000000000000004e-07 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116511+0000", 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0730000000000005e-06 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116533+0000", 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1803000000000001e-05 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116549+0000", 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6348999999999999e-05 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133399+0000", 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016849553999999999 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.133748+0000", 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034938299999999999 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.134943+0000", 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011943699999999999 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146326+0000", 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011383760999999999 2024-04-18T22:23:22.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154867+0000", 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085406530000000005 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154981+0000", 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114341 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155074+0000", 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2770999999999997e-05 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:29 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.764930008999997, 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038388247, 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116568+0000", 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116569+0000", 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2160000000000001e-06 2024-04-18T22:23:22.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116577+0000", 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116595+0000", 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8476000000000001e-05 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116610+0000", 2024-04-18T22:23:22.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4175999999999999e-05 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135127+0000", 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018517648000000001 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135323+0000", 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605600000000001 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135658+0000", 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033513 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146342+0000", 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010683405999999999 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154701+0000", 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083594749999999999 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154843+0000", 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014191200000000001 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.154956+0000", 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113142 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:35 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.764870262000002, 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038572724000000003, 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116628+0000", 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116629+0000", 2024-04-18T22:23:22.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116652+0000", 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3319000000000002e-05 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116672+0000", 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9949000000000002e-05 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116689+0000", 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6382999999999999e-05 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128403+0000", 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011714218 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.128753+0000", 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035047300000000002 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.129077+0000", 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032312399999999999 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.138480+0000", 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094034779999999998 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155032+0000", 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016552015999999999 2024-04-18T22:23:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155125+0000", 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2453000000000004e-05 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155201+0000", 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6003999999999999e-05 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:22 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.764790192, 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038714272000000001, 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116708+0000", 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116709+0000", 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1999999999999999e-06 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116717+0000", 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6229999999999997e-06 2024-04-18T22:23:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116734+0000", 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6835999999999999e-05 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116751+0000", 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7465999999999998e-05 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.135749+0000", 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018998417 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136096+0000", 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034635900000000002 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.136415+0000", 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031908799999999998 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.146478+0000", 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010062715999999999 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155192+0000", 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087140589999999997 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155331+0000", 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013958500000000001 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.155422+0000", 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0922999999999997e-05 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:26 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.764728783999999, 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023789055999999999, 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116769+0000", 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116771+0000", 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.443e-06 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116777+0000", 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116794+0000", 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7316000000000002e-05 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116808+0000", 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3763e-05 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116965+0000", 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015681000000000001 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116980+0000", 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.120921+0000", 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039406199999999997 2024-04-18T22:23:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.121807+0000", 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088550100000000002 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.140558+0000", 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018751799999999999 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(mds.0.3:32 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.764644470999997, 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032998882, 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116854+0000", 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116855+0000", 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.297e-06 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116878+0000", 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3405999999999999e-05 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116929+0000", 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1193999999999998e-05 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.116945+0000", 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123157+0000", 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006212904 2024-04-18T22:23:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.123633+0000", 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047570400000000002 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.149852+0000", 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026219281000000001 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(mds.0.3:23 3.d e17/15 3:b50e409b:::mds_snaptable:head v 17'1, mlcod=0'0)", 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.757488737999999, 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018502906, 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124009+0000", 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124010+0000", 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124029+0000", 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8488999999999999e-05 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124082+0000", 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3387999999999998e-05 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.124107+0000", 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4839000000000001e-05 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131380+0000", 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072729889999999997 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.131424+0000", 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4301000000000002e-05 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142406+0000", 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010982277 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:22:32.142512+0000", 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010590900000000001 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25664 5.f 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.078986371999999, 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093405780999999993, 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25664 2024-04-18T22:23:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802512+0000", 2024-04-18T22:23:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802645+0000", 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013290900000000001 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802656+0000", 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-05 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802661+0000", 2024-04-18T22:23:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6729999999999997e-06 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802711+0000", 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9329000000000002e-05 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802804+0000", 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2929000000000004e-05 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802894+0000", 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0346999999999994e-05 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.807400+0000", 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045058290000000003 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895676+0000", 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088275705999999995 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895810+0000", 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134635 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895917+0000", 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107309 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25666 5.f 5:f7add380:::benchmark_data_smithi032_22800_object25665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.078812854999999, 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093393754999999995, 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25666 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802685+0000", 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802686+0000", 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802716+0000", 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9954000000000002e-05 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802723+0000", 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5689999999999999e-06 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.802728+0000", 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6079999999999998e-06 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803716+0000", 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098778600000000005 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.803871+0000", 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015491100000000001 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.804071+0000", 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020055600000000001 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.810020+0000", 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059493339999999997 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.895982+0000", 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085962083999999994 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896036+0000", 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3813999999999997e-05 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.896079+0000", 2024-04-18T22:23:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2676000000000003e-05 2024-04-18T22:23:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25673 5.d 5:becd3ab4:::benchmark_data_smithi032_22800_object25672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.073445338999999, 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085178675999999995, 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25673 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808053+0000", 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808054+0000", 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2900000000000002e-07 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808114+0000", 2024-04-18T22:23:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9993e-05 2024-04-18T22:23:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808126+0000", 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808136+0000", 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.064e-05 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808186+0000", 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9471000000000001e-05 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808277+0000", 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0906e-05 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.808387+0000", 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011026099999999999 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.812706+0000", 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043185289999999998 2024-04-18T22:23:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.892987+0000", 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080281292000000004 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893126+0000", 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013872099999999999 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893231+0000", 2024-04-18T22:23:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010582300000000001 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25675 5.f 5:f60cbd32:::benchmark_data_smithi032_22800_object25674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.066003768000002, 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082348202999999995, 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25675 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815494+0000", 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815495+0000", 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815546+0000", 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0933999999999997e-05 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815572+0000", 2024-04-18T22:23:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5936999999999999e-05 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815585+0000", 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2860000000000001e-05 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815664+0000", 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9334999999999999e-05 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.815838+0000", 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017410700000000001 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.816021+0000", 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018247599999999999 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.817763+0000", 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001742494 2024-04-18T22:23:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897556+0000", 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079792478999999999 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897717+0000", 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161199 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.897843+0000", 2024-04-18T22:23:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000125415 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:25682 5.b 5:d19b5078:::benchmark_data_smithi032_22800_object25681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.987686406999998, 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017001512999999999, 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25682 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893812+0000", 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4899999999999995e-07 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893846+0000", 2024-04-18T22:23:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3546e-05 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893855+0000", 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5689999999999994e-06 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.893862+0000", 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7070000000000003e-06 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898400+0000", 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045379089999999997 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898589+0000", 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188965 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.898831+0000", 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000242181 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.902115+0000", 2024-04-18T22:23:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032839369999999998 2024-04-18T22:23:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910719+0000", 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086036350000000001 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910771+0000", 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2584999999999999e-05 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:05.910813+0000", 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1829999999999998e-05 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:56088 5.0 5:0839553e:::benchmark_data_smithi032_22800_object56087:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:19.055580+0000", 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.8259177700000002, 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017740243999999999, 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 56088 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055580+0000", 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055580+0000", 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055581+0000", 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0800000000000003e-07 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055721+0000", 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013982999999999999 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055730+0000", 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3340000000000008e-06 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.055748+0000", 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7776000000000001e-05 2024-04-18T22:23:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.064792+0000", 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090440720000000002 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.064933+0000", 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014087099999999999 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.065100+0000", 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016689 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.069278+0000", 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004177648 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.073223+0000", 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039457870000000001 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.073277+0000", 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3899e-05 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.073321+0000", 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3229000000000002e-05 2024-04-18T22:23:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:57190 5.e 5:70cb011b:::benchmark_data_smithi032_22800_object57189:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:19.579775+0000", 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.3017226800000001, 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016862696, 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57190 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.579775+0000", 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.579775+0000", 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.579776+0000", 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0500000000000002e-07 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.579845+0000", 2024-04-18T22:23:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9120000000000002e-05 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.579854+0000", 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9760000000000001e-06 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.579862+0000", 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4109999999999999e-06 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.584252+0000", 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043901399999999998 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.584398+0000", 2024-04-18T22:23:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014656099999999999 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.584520+0000", 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121743 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.588936+0000", 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044154119999999996 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.596514+0000", 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075785280000000002 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.596585+0000", 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0702000000000002e-05 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:19.596638+0000", 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3297999999999997e-05 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:58669 5.4 5:266c9129:::benchmark_data_smithi032_22800_object58668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:20.298924+0000", 2024-04-18T22:23:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.5825739080000001, 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016894064, 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 58669 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.298924+0000", 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.298924+0000", 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.298925+0000", 2024-04-18T22:23:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.299004+0000", 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9579999999999994e-05 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.299014+0000", 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2520000000000005e-06 2024-04-18T22:23:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.299020+0000", 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9549999999999997e-06 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.303995+0000", 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049755570000000002 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.304093+0000", 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7397000000000003e-05 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.304222+0000", 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129054 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.309656+0000", 2024-04-18T22:23:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347140000000002 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.315747+0000", 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060904690000000003 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.315790+0000", 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2996e-05 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:20.315818+0000", 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8402999999999999e-05 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:63499 5.8 5:1b80edc0:::benchmark_data_smithi032_22800_object63498:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:22.661059+0000", 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.22043886400000001, 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017587042000000001, 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63499 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.661059+0000", 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.661059+0000", 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.661060+0000", 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.661103+0000", 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3408000000000002e-05 2024-04-18T22:23:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.661111+0000", 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1140000000000006e-06 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.661119+0000", 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1779999999999997e-06 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.666299+0000", 2024-04-18T22:23:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051801010000000003 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.666449+0000", 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150125 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.666606+0000", 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015717699999999999 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.676479+0000", 2024-04-18T22:23:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098725970000000003 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.678537+0000", 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020587829999999998 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.678594+0000", 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6696000000000001e-05 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.678646+0000", 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2178000000000001e-05 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:63588 5.9 5:915a0d22:::benchmark_data_smithi032_22800_object63587:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:22.704229+0000", 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.17726937800000001, 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019626881999999998, 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63588 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.704229+0000", 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.704229+0000", 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.704229+0000", 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-07 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.704258+0000", 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8924e-05 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.704264+0000", 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5430000000000001e-06 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.704267+0000", 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5300000000000001e-06 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.713458+0000", 2024-04-18T22:23:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091907050000000004 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.713598+0000", 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000140097 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.713737+0000", 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000139018 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.720903+0000", 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071658650000000004 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.723800+0000", 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002896746 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-04-18T22:23:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.723828+0000", 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8836999999999999e-05 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:22.723856+0000", 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7228000000000001e-05 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:23.002 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:23.003 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:23.003+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:23.004 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:23:23.005 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:23.103 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:23.102+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:23.156 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:23.156 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:23.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64483 5.9 e33/23 5:94a911a5:::benchmark_data_smithi032_22800_object64482:head v 33'4021, mlcod=33'4019)", 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.151522+0000", 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0056356410000000003, 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0056606290000000004, 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151522+0000", 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151522+0000", 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151523+0000", 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2300000000000001e-07 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151583+0000", 2024-04-18T22:23:23.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9732999999999999e-05 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151593+0000", 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7180000000000008e-06 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151598+0000", 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.801e-06 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151646+0000", 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7775999999999998e-05 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151667+0000", 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0468000000000001e-05 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64485 5.7 e33/23 5:e23c3e70:::benchmark_data_smithi032_22800_object64484:head v 33'4050, mlcod=33'4048)", 2024-04-18T22:23:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.151891+0000", 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.005266794, 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0053592559999999997, 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151891+0000", 2024-04-18T22:23:23.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151891+0000", 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151892+0000", 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3200000000000005e-07 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151919+0000", 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7416e-05 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151929+0000", 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7699999999999996e-06 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.151945+0000", 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5497000000000001e-05 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152000+0000", 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5358000000000001e-05 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152022+0000", 2024-04-18T22:23:23.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1844e-05 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64487 5.2 e33/23 5:4d69c492:::benchmark_data_smithi032_22800_object64486:head v 33'4035, mlcod=33'4031)", 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.152603+0000", 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0045553210000000002, 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0047092109999999996, 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152603+0000", 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152603+0000", 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152603+0000", 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9999999999999998e-07 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152685+0000", 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1724999999999994e-05 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152696+0000", 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1355000000000001e-05 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.152703+0000", 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7780000000000002e-06 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153062+0000", 2024-04-18T22:23:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00035917799999999998 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153080+0000", 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7691999999999999e-05 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.169 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64491 5.d e33/19 5:b7b5f2a3:::benchmark_data_smithi032_22800_object64490:head v 33'4013, mlcod=33'4011)", 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.153260+0000", 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0038977870000000002, 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040945579999999999, 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153260+0000", 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153260+0000", 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153261+0000", 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153300+0000", 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9314999999999999e-05 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153311+0000", 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0241000000000001e-05 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153318+0000", 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1169999999999998e-06 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153394+0000", 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6255999999999996e-05 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.153413+0000", 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8887000000000001e-05 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64495 5.9 e33/23 5:9158311b:::benchmark_data_smithi032_22800_object64494:head v 33'4022, mlcod=33'4019)", 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.154694+0000", 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0024637370000000001, 2024-04-18T22:23:23.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026993329999999999, 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154694+0000", 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154694+0000", 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154695+0000", 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3099999999999998e-07 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154748+0000", 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3155999999999997e-05 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154757+0000", 2024-04-18T22:23:23.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8240000000000006e-06 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154762+0000", 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6759999999999998e-06 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154808+0000", 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6448e-05 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.154832+0000", 2024-04-18T22:23:23.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3524e-05 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64493 5.1 e33/23 5:833177ed:::benchmark_data_smithi032_22800_object64492:head v 33'3980, mlcod=33'3978)", 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.155969+0000", 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001189212, 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014548510000000001, 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.155969+0000", 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.155969+0000", 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.155970+0000", 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0200000000000002e-07 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156017+0000", 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7261e-05 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156036+0000", 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8972999999999999e-05 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156041+0000", 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7960000000000003e-06 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156097+0000", 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6915e-05 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156120+0000", 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2466000000000001e-05 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64488 5.2 e33/23 5:4da3b989:::benchmark_data_smithi032_22800_object64487:head v 33'4036, mlcod=33'4032)", 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.156060+0000", 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0010980149999999999, 2024-04-18T22:23:23.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001393254, 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156060+0000", 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156060+0000", 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156060+0000", 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6399999999999998e-07 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156096+0000", 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6090999999999997e-05 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156110+0000", 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3638000000000001e-05 2024-04-18T22:23:23.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156114+0000", 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6399999999999999e-06 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156157+0000", 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3296e-05 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156176+0000", 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8858000000000001e-05 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64494 5.6 e33/19 5:6f4fbb80:::benchmark_data_smithi032_22800_object64493:head v 33'4011, mlcod=33'4009)", 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.156123+0000", 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0010351430000000001, 2024-04-18T22:23:23.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0013660040000000001, 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156123+0000", 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156123+0000", 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156123+0000", 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9499999999999998e-07 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156251+0000", 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012819 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156261+0000", 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7990000000000005e-06 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156267+0000", 2024-04-18T22:23:23.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5470000000000004e-06 2024-04-18T22:23:23.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156316+0000", 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9579000000000001e-05 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156342+0000", 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5605e-05 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64498 5.3 e33/19 5:cadda535:::benchmark_data_smithi032_22800_object64497:head v 33'4200, mlcod=33'4198)", 2024-04-18T22:23:23.194 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.156283+0000", 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00087490699999999996, 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012445030000000001, 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156283+0000", 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156283+0000", 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156284+0000", 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9799999999999999e-07 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156319+0000", 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5580999999999997e-05 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156335+0000", 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5576000000000001e-05 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156340+0000", 2024-04-18T22:23:23.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3770000000000004e-06 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156405+0000", 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5304999999999999e-05 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.156422+0000", 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6385e-05 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:64497 5.1 e33/23)", 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:23.157234+0000", 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9999242, 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032306700000000002, 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.157234+0000", 2024-04-18T22:23:23.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.157234+0000", 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.157234+0000", 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7000000000000005e-07 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.157286+0000", 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1631000000000001e-05 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.157295+0000", 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4530000000000008e-06 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:23.157301+0000", 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8220000000000003e-06 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.202 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:23.202 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 10 2024-04-18T22:23:23.202 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:23.202 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 27 16 64579 64563 149.435 126.312 0.0112338 0.00668671 2024-04-18T22:23:23.203 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:23.204 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:23.203+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:23.305 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:23.303+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:23.352 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:23.352 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:23.353 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:23.353 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:23.353 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:23.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:23.406 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:23.406+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:23.512 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:23.506+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:23.564 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:23.564 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:23.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:23.565 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:23.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.565 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.565 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:23.565 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.449114143999999, 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:23.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:23.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:23.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:23.573 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.449045546000001, 2024-04-18T22:23:23.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:23.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:23.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:23.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:23.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:23.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:23.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.4489497, 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:23.584 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:23.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:23.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:23.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:23.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:23.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:23.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:23.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448892817999997, 2024-04-18T22:23:23.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:23.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:23.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:23.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:23.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:23.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:23.602 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448833145999998, 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.603 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:23.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:23.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:23.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:23.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:23.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:23.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:23.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:23.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.612 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448775183999999, 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.615 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.616 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:23.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.618 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.619 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:23.619 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:23.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.619 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.620 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:23.620 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:23.620 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.621 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448555491999997, 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.622 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.624 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:23.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.626 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:23.627 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:23.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448498563000001, 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:23.631 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:23.632 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.633 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:23.634 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:23.636 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448402676999997, 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:23.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:23.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:23.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.644 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448340348999999, 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:23.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:23.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:23.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:23.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.651 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:23.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:23.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448267870999999, 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.657 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:23.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448115403000003, 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:23.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.670 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.448036680000001, 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:23.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:23.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.446515824000002, 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.677 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:23.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:23.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 51.446417472, 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.685 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:23.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:23.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.764695519, 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:23.691 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:23.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.764041305999999, 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:23.697 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:23.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:23.701 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.763934611, 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.762068702000001, 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.706 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.710 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.761403379000001, 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:23.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:23.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:23.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:23.715 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:23.716 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:23.717 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:23.654+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:23.762 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:23.762+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:23.860 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:23.859+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:23.876 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:23.876 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:23.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.876 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65979 5.5 5:adbb90de:::benchmark_data_smithi032_22800_object65978:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.876 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.871622+0000", 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0029096619999999999, 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029372299999999999, 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65979 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871622+0000", 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871622+0000", 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871623+0000", 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7700000000000004e-07 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871649+0000", 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5888999999999999e-05 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871668+0000", 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9086999999999999e-05 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871678+0000", 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6600000000000007e-06 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873690+0000", 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020125709999999999 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873820+0000", 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129944 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873964+0000", 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014382300000000001 2024-04-18T22:23:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65981 5.5 5.6b032c5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.872142+0000", 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023901640000000002, 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024904010000000002, 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.881 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65981 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872142+0000", 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872142+0000", 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872143+0000", 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0249999999999999e-06 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872207+0000", 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.436e-05 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872214+0000", 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2470000000000002e-06 2024-04-18T22:23:23.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872222+0000", 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2069999999999998e-06 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65982 5.a 5.feb0127a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.872231+0000", 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023006799999999998, 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002450696, 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65982 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872231+0000", 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872231+0000", 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872232+0000", 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0200000000000003e-07 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872305+0000", 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3689999999999994e-05 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872312+0000", 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2759999999999997e-06 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872317+0000", 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4700000000000001e-06 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:65980 5.8 e33/19 5:1de47f9b:::benchmark_data_smithi032_22800_object65979:head v 33'4065, mlcod=33'4062)", 2024-04-18T22:23:23.887 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.872328+0000", 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002204368, 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002404846, 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872328+0000", 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872328+0000", 2024-04-18T22:23:23.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872328+0000", 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9400000000000005e-07 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872376+0000", 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7345000000000002e-05 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872382+0000", 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5050000000000004e-06 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872386+0000", 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1200000000000004e-06 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872524+0000", 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000137885 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872536+0000", 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.203e-05 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:65973 5.5 e33/19)", 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.872420+0000", 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002111702, 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002363305, 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872420+0000", 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872420+0000", 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872421+0000", 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3599999999999999e-07 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872424+0000", 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7810000000000001e-06 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872431+0000", 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8019999999999992e-06 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872436+0000", 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6990000000000002e-06 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:65984 5.e e33/19 5:7a7d2447:::benchmark_data_smithi032_22800_object65983:head v 33'4045, mlcod=33'4043)", 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.872806+0000", 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001725919, 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020217590000000001, 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872806+0000", 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872806+0000", 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872807+0000", 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4799999999999998e-07 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872865+0000", 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8699000000000002e-05 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872873+0000", 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0690000000000002e-06 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872879+0000", 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6559999999999997e-06 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872939+0000", 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0040000000000001e-05 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872956+0000", 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6653000000000001e-05 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65985 5.5 5.82f27245 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.872424+0000", 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021081030000000001, 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024555319999999999, 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65985 2024-04-18T22:23:23.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872424+0000", 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872424+0000", 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.872424+0000", 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7100000000000002e-07 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873693+0000", 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012689870000000001 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873701+0000", 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7340000000000008e-06 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873708+0000", 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9600000000000003e-06 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65987 5.2 5:4e7a0592:::benchmark_data_smithi032_22800_object65986:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.901 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.873723+0000", 2024-04-18T22:23:23.902 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00080904399999999997, 2024-04-18T22:23:23.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001210432, 2024-04-18T22:23:23.902 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65987 2024-04-18T22:23:23.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873723+0000", 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873723+0000", 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873724+0000", 2024-04-18T22:23:23.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.61e-07 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873751+0000", 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6961000000000002e-05 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873757+0000", 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6420000000000001e-06 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873763+0000", 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6949999999999997e-06 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873828+0000", 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5318e-05 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873901+0000", 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2724000000000004e-05 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.873976+0000", 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5240999999999993e-05 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:65986 5.b e33/19 5:d892bafe:::benchmark_data_smithi032_22800_object65985:head v 33'4169, mlcod=33'4167)", 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.874808+0000", 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997239, 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022338699999999999, 2024-04-18T22:23:23.910 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874808+0000", 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874808+0000", 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874809+0000", 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8500000000000002e-07 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874840+0000", 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1748000000000002e-05 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874847+0000", 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7399999999999998e-06 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874852+0000", 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9969999999999998e-06 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874903+0000", 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1011999999999999e-05 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874923+0000", 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9448000000000001e-05 2024-04-18T22:23:23.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:65977 5.a e33/19)", 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.874873+0000", 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9996591, 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220387, 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874873+0000", 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874873+0000", 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874873+0000", 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-07 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874881+0000", 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.892e-06 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874900+0000", 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8303000000000001e-05 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.874906+0000", 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.054e-06 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65973 5.5 5:aff9505a:::benchmark_data_smithi032_22800_object65972:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.867279+0000", 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0072527570000000003, 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0078699249999999998, 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65973 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867279+0000", 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867279+0000", 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867280+0000", 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1399999999999997e-07 2024-04-18T22:23:23.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867317+0000", 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7456999999999997e-05 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867326+0000", 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0119999999999994e-06 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867332+0000", 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.626e-06 2024-04-18T22:23:23.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867427+0000", 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5138999999999998e-05 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867554+0000", 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012669999999999999 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.867675+0000", 2024-04-18T22:23:23.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121147 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870485+0000", 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028103239999999999 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:65974 5.8 e33/19 5:161b1a5b:::benchmark_data_smithi032_22800_object65973:head v 33'4064, mlcod=33'4061)", 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.870521+0000", 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0040106430000000004, 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047057469999999997, 2024-04-18T22:23:23.921 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870521+0000", 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870521+0000", 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870522+0000", 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.51e-07 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870567+0000", 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4941000000000002e-05 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870574+0000", 2024-04-18T22:23:23.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.362e-06 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870580+0000", 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0360000000000003e-06 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870618+0000", 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7564000000000002e-05 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.870629+0000", 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1447000000000001e-05 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65977 5.a 5:56526c84:::benchmark_data_smithi032_22800_object65976:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.871127+0000", 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0034051950000000002, 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041579360000000001, 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.925 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65977 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871127+0000", 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871127+0000", 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871142+0000", 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4853e-05 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871200+0000", 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8406999999999998e-05 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871225+0000", 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5412e-05 2024-04-18T22:23:23.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871231+0000", 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7989999999999999e-06 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871281+0000", 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9546000000000003e-05 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871411+0000", 2024-04-18T22:23:23.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012984100000000001 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871542+0000", 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130915 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:65978 5.5 5:afe82195:::benchmark_data_smithi032_22800_object65977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:23.871252+0000", 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032801029999999999, 2024-04-18T22:23:23.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041008870000000001, 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 65978 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871252+0000", 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871252+0000", 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871252+0000", 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4300000000000003e-07 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871284+0000", 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1581999999999997e-05 2024-04-18T22:23:23.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871291+0000", 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9e-06 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871297+0000", 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9499999999999998e-06 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871349+0000", 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1656000000000001e-05 2024-04-18T22:23:23.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871462+0000", 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113767 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:23.871603+0000", 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014041999999999999 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:23.933 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 14 2024-04-18T22:23:23.934 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:23.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:23.959 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:23.959+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:24.060 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:24.059+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:24.125 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:24.125 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:24.125 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:24.125 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:24.125 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:24.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:24.162 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:24.159+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:24.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 28 16 66599 66583 148.607 126.25 0.0082568 0.00672388 2024-04-18T22:23:24.266 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:24.262+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183918845999997, 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:24.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:24.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183786353999999, 2024-04-18T22:23:24.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:24.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:24.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:24.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:24.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.312 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183714768000002, 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:24.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:24.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183586515000002, 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:24.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:24.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183477187999998, 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:24.330 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183220851000002, 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:24.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183133365000003, 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.183055379000002, 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:24.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.500768297, 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.345 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:24.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:24.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:24.350 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.499111918000001, 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:24.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:24.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.498987200999998, 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:24.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.497207542999998, 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.362 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:24.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:24.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:24.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.496724872000001, 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:24.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:24.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:24.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:24.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.496485796999998, 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:24.373 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:24.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:24.378 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.496365025999999, 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.379 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.380 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.381 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.383 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:24.384 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.493475614000001, 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:24.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.386 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.387 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.388 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.390 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.493353795000001, 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.393 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.493243346, 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.396 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:24.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.398 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.400 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.493173878, 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.402 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:24.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:24.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:24.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:24.406 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.493116737000001, 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:24.407 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.412 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:24.413 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:24.413 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:24.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:23:24.415 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:24.410+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:24.515 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:24.514+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:24.569 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:67340 5.e 5:7434e9d6:::benchmark_data_smithi032_22800_object67339:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.569707+0000", 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0014741940000000001, 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014939020000000001, 2024-04-18T22:23:24.570 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67340 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569707+0000", 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569707+0000", 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569708+0000", 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0189999999999999e-06 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569796+0000", 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7379000000000005e-05 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569810+0000", 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3944999999999999e-05 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569818+0000", 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1259999999999998e-06 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569872+0000", 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3903999999999998e-05 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.570008+0000", 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013610900000000001 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.570166+0000", 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015779999999999999 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:67341 5.0 5:0349a4e0:::benchmark_data_smithi032_22800_object67340:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:24.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.569837+0000", 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0013442180000000001, 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014141030000000001, 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67341 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569837+0000", 2024-04-18T22:23:24.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569837+0000", 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569838+0000", 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2500000000000005e-07 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569876+0000", 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7926999999999998e-05 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569882+0000", 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1759999999999998e-06 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569888+0000", 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9059999999999999e-06 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:24.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569934+0000", 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5799e-05 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.570018+0000", 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4456000000000006e-05 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.570130+0000", 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111595 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4586.0:67327 5.0 e33/19)", 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.569921+0000", 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0012606259999999999, 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013698429999999999, 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569921+0000", 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569921+0000", 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569921+0000", 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569925+0000", 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3349999999999998e-06 2024-04-18T22:23:24.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569932+0000", 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000001e-06 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569938+0000", 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6010000000000002e-06 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4586.0:67325 5.8 e33/19)", 2024-04-18T22:23:24.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.569952+0000", 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0012295299999999999, 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013631909999999999, 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569952+0000", 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569952+0000", 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569952+0000", 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.15e-07 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569955+0000", 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0920000000000002e-06 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569962+0000", 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8419999999999999e-06 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.569967+0000", 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0370000000000002e-06 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:67325 5.8 5:1ac0ddcc:::benchmark_data_smithi032_22800_object67324:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.561125+0000", 2024-04-18T22:23:24.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010056109000000001, 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010222518999999999, 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67325 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.561125+0000", 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.561125+0000", 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.561126+0000", 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6400000000000002e-07 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.561186+0000", 2024-04-18T22:23:24.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9621999999999999e-05 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.561193+0000", 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7600000000000002e-06 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.561201+0000", 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2250000000000003e-06 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:24.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.562424+0000", 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012233330000000001 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.562571+0000", 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014744899999999999 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.562746+0000", 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000175062 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:24.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566234+0000", 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034875370000000002 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:67327 5.0 5:0b453c39:::benchmark_data_smithi032_22800_object67326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.564105+0000", 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0070768080000000004, 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072879599999999996, 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:24.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67327 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564105+0000", 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564105+0000", 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564105+0000", 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564182+0000", 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6927000000000004e-05 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564192+0000", 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1030000000000008e-06 2024-04-18T22:23:24.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564196+0000", 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.673e-06 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564225+0000", 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8537000000000001e-05 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564303+0000", 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8396e-05 2024-04-18T22:23:24.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.564385+0000", 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1575000000000004e-05 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566236+0000", 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001851669 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.593 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:67328 5.e 5:76da6857:::benchmark_data_smithi032_22800_object67327:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.565562+0000", 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0056192350000000002, 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058741510000000002, 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67328 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565562+0000", 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565562+0000", 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565563+0000", 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565671+0000", 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108644 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565679+0000", 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7219999999999999e-06 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565684+0000", 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3919999999999996e-06 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565786+0000", 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102388 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.565914+0000", 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000127713 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566053+0000", 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013924599999999999 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.568381+0000", 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023276130000000001 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4586.0:67333 5.0 5:0ab1333d:::benchmark_data_smithi032_22800_object67332:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-04-18T22:23:24.566494+0000", 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0046872210000000001, 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049886649999999998, 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-04-18T22:23:24.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4586", 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67333 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566494+0000", 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-04-18T22:23:24.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566494+0000", 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566495+0000", 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999998e-07 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566554+0000", 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8924e-05 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566563+0000", 2024-04-18T22:23:24.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5990000000000008e-06 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566568+0000", 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9629999999999997e-06 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566616+0000", 2024-04-18T22:23:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7172999999999997e-05 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566735+0000", 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119336 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-04-18T22:23:24.566867+0000", 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013234 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-18T22:23:24.603 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-18T22:23:24.604 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 8 2024-04-18T22:23:24.604 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:24.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T22:23:24.651 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:24.650+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:24.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:24.717 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:24.718+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:24.760 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:23:24.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:23:24.760 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-04-18T22:23:24.760 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-04-18T22:23:24.760 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:23:24.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T22:23:24.823 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:24.822+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:24.854 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:24.874 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:24.896 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:24.919 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:24.918+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:24.980 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:24.980 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:25.001 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:25.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:23:25.047 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:25.047+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:25.120 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:25.119+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:25.157 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:25.158 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:25.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:25.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 29 16 68633 68617 147.865 127.125 0.00542528 0.00675797 2024-04-18T22:23:25.221 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:25.218+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:25.322 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:25.322+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:25.365 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:25.365 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:25.383 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:25.430 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:25.431+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:25.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:25.523+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:25.524 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:25.524 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:25.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:25.625 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:23:25.622+0000 7fe88b250640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 21629) UID: 0 2024-04-18T22:23:25.687 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:25.687 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:25.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:25.749 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:25.750+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:25.826 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:25.826+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:25.849 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:25.849 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:25.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:25.940 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2024-04-18T22:23:25.940 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:25.941 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2] 2024-04-18T22:23:25.941 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.1 doing powercycle of ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:23:25.941 INFO:teuthology.orchestra.console.smithi096:Power off 2024-04-18T22:23:25.941 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T22:23:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:23:25.957 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:25.958+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:25.964 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2024-04-18T22:23:25.964 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T22:23:25.987 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-04-18T22:23:26.029 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:26.027+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:26.062 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:26.063 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:26.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:26.134 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:26.135+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:26.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 30 16 70506 70490 146.838 117.062 0.00669139 0.00680553 2024-04-18T22:23:26.229 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:26.229 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:26.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:26.231+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:26.250 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:26.331 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:26.330+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:26.431 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:26.431+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:26.434 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:26.434 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:26.452 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:26.533 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:26.530+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:26.633 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:26.633 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:26.634 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:26.631+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:26.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:26.736 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:26.734+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:26.800 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:26.800 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:26.818 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:26.861 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:26.862+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:26.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:26.937 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:26.935+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:27.008 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:27.008 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:27.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:27.069 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:27.070+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:27.079 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:27.101 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:27.120 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:27.139 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:27.139+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:27.174 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:27.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:27.174 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:27.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058832054, 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:27.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.180 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058699562000001, 2024-04-18T22:23:27.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:27.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:27.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:27.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058627975999997, 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:27.193 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058499722999997, 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:27.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:27.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:27.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:27.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:27.199 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058390396, 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:27.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058134058999997, 2024-04-18T22:23:27.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.058046572999999, 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:27.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 55.057968586999998, 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:27.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.375681504999999, 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:27.219 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.224 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.374025125999999, 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:27.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:27.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.373900409000001, 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:27.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:27.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:27.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.372120751000001, 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:27.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:27.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.37163808, 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:27.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.371399005000001, 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.371278233999998, 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.253 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:27.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:27.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:27.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:27.258 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.368388822, 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:27.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:27.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:27.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:27.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.368267003, 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.264 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:27.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:27.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.368156553999999, 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.368087085999999, 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:27.280 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.368029945, 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:27.281 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:27.286 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:27.287 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 31 16 72411 72395 145.942 119.062 0.00347809 0.00684681 2024-04-18T22:23:27.288 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:27.287+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:27.288 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:27.389 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:27.386+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:27.431 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:72886 5.5 e34/19 5:ae5aa69a:::benchmark_data_smithi032_22800_object72885:head v 34'4487, mlcod=34'4485)", 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:27.429500+0000", 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003451567, 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034668569999999998, 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429500+0000", 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429500+0000", 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429501+0000", 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7999999999999995e-07 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429542+0000", 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1517000000000002e-05 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429551+0000", 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1620000000000008e-06 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429559+0000", 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8230000000000001e-06 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429611+0000", 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1422999999999999e-05 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.429636+0000", 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5097999999999999e-05 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:72889 5.9 e34/23 5:988110fc:::benchmark_data_smithi032_22800_object72888:head v 34'4570, mlcod=34'4568)", 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:27.430716+0000", 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0022355859999999999, 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022872359999999998, 2024-04-18T22:23:27.436 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430716+0000", 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430716+0000", 2024-04-18T22:23:27.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430717+0000", 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9699999999999996e-07 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430784+0000", 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7207999999999998e-05 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430792+0000", 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8089999999999993e-06 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430799+0000", 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430849+0000", 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0775000000000001e-05 2024-04-18T22:23:27.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:27.430867+0000", 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7971000000000001e-05 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 2 2024-04-18T22:23:27.440 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:27.451 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:27.490 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:27.490+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:27.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:27.591+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:27.591 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:27.591 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:27.592 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:27.592 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:27.592 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:27.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:27.691 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:27.690+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:27.792 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:27.791+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:27.801 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:27.801 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:27.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.686096847999998, 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:27.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:27.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:27.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:27.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.68602825, 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685932403999999, 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685875522000003, 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.817 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:27.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:27.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:27.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685815849999997, 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.822 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:27.827 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685757887999998, 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:27.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:27.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:27.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.832 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685538196000003, 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:27.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:27.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:27.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685481267, 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685385381000003, 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.843 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:27.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:27.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:27.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:27.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685323052999998, 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:27.848 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:27.853 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685250574999998, 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:27.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:27.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.858 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685098107000002, 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:27.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:27.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:27.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:27.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.685019384, 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:27.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.683498528000001, 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:27.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 55.683400175999999, 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 22.001678222999999, 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:27.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.881 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 22.001024009999998, 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:27.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:27.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:27.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.885 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 22.000917314999999, 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:27.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.999051406, 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.998386083, 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:27.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:27.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:27.898 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:27.899 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:27.898+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:27.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:27.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:23:28.000 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:27.999+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74105 5.3 5:c7e12c44:::benchmark_data_smithi032_22800_object74104:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.047699+0000", 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0088018309999999995, 2024-04-18T22:23:28.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0088296610000000008, 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74105 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047699+0000", 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047699+0000", 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047699+0000", 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2900000000000002e-07 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047758+0000", 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8114e-05 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047766+0000", 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7569999999999993e-06 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047773+0000", 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.212e-06 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056075+0000", 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083028879999999996 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056230+0000", 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000154375 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056385+0000", 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015481800000000001 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.060 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74086 5.7 5:eb028a01:::benchmark_data_smithi032_22800_object74085:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.037276+0000", 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.019223911, 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.01933346, 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74086 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.037276+0000", 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.037276+0000", 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.037277+0000", 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.850000000000001e-07 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.037315+0000", 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7224999999999997e-05 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.037325+0000", 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.025e-05 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.037334+0000", 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1749999999999994e-06 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.039973+0000", 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026386740000000001 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040128+0000", 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015481800000000001 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040215+0000", 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7089000000000006e-05 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043320+0000", 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003105174 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74106 5.6 5:62eccafa:::benchmark_data_smithi032_22800_object74105:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.047892+0000", 2024-04-18T22:23:28.065 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0086081249999999995, 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0087931529999999997, 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74106 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047892+0000", 2024-04-18T22:23:28.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047892+0000", 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047893+0000", 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8999999999999997e-07 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047959+0000", 2024-04-18T22:23:28.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6258000000000002e-05 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047969+0000", 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8050000000000001e-06 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047975+0000", 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1330000000000004e-06 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056072+0000", 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0080974459999999995 2024-04-18T22:23:28.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056223+0000", 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015102799999999999 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056372+0000", 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014900100000000001 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74107 5.5 5.1ca264b5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.047990+0000", 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0085100060000000005, 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0087592820000000002, 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74107 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047990+0000", 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047990+0000", 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047991+0000", 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0299999999999999e-07 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048067+0000", 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5944000000000003e-05 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048077+0000", 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0659000000000001e-05 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048083+0000", 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0000000000000002e-06 2024-04-18T22:23:28.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:74098 5.5 e34/19)", 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.048082+0000", 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0084186160000000003, 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.008715202, 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048082+0000", 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048082+0000", 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048082+0000", 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8899999999999999e-07 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048086+0000", 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5190000000000001e-06 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048094+0000", 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6599999999999995e-06 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048099+0000", 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7139999999999998e-06 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.076 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:74097 5.3 e34/19)", 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.048192+0000", 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0083079420000000005, 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0086442480000000002, 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048192+0000", 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048192+0000", 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048193+0000", 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5900000000000002e-07 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048196+0000", 2024-04-18T22:23:28.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6450000000000001e-06 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048202+0000", 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5860000000000001e-06 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.048207+0000", 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0030000000000002e-06 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74111 5.5 5.d6f5b5f5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.055724+0000", 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00077590500000000002, 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011541279999999999, 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74111 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.080 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.055724+0000", 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.055724+0000", 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.055725+0000", 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2399999999999996e-07 2024-04-18T22:23:28.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.055799+0000", 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3807999999999995e-05 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.055810+0000", 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0429e-05 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.055817+0000", 2024-04-18T22:23:28.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2869999999999999e-06 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:74112 5.8 e34/19 5:1e245265:::benchmark_data_smithi032_22800_object74111:head v 34'4534, mlcod=34'4532)", 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.056159+0000", 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00034165900000000002, 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00076942899999999997, 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:28.083 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056159+0000", 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056159+0000", 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056159+0000", 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.68e-07 2024-04-18T22:23:28.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056216+0000", 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6909e-05 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056225+0000", 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7620000000000001e-06 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056231+0000", 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7760000000000003e-06 2024-04-18T22:23:28.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056296+0000", 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5680999999999997e-05 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056315+0000", 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8573999999999999e-05 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74090 5.5 5:a56555fd:::benchmark_data_smithi032_22800_object74089:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.039933+0000", 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.016566906999999999, 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017054337999999999, 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74090 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.039933+0000", 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.039933+0000", 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.039934+0000", 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.850000000000001e-07 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040100+0000", 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016527499999999999 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040110+0000", 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0715e-05 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040118+0000", 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7640000000000005e-06 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040270+0000", 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015216200000000001 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040404+0000", 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000134136 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.040581+0000", 2024-04-18T22:23:28.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000176393 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:74096 5.b e34/19 5:dbb7d3b6:::benchmark_data_smithi032_22800_object74095:head v 34'4648, mlcod=34'4645)", 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.043117+0000", 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.013383058999999999, 2024-04-18T22:23:28.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013936041, 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043117+0000", 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043117+0000", 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043118+0000", 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.91e-07 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043144+0000", 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6908000000000001e-05 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043160+0000", 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5038000000000001e-05 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043165+0000", 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2000000000000002e-06 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045017+0000", 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001852753 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045037+0000", 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9063999999999999e-05 2024-04-18T22:23:28.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74097 5.3 5:c1a0bcfe:::benchmark_data_smithi032_22800_object74096:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.043237+0000", 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.013263161000000001, 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013873837999999999, 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74097 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043237+0000", 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043237+0000", 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043238+0000", 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.78e-07 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043285+0000", 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6752000000000002e-05 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043295+0000", 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0508e-05 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043302+0000", 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6000000000000003e-06 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045005+0000", 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017027940000000001 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045122+0000", 2024-04-18T22:23:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117599 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045244+0000", 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012208600000000001 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74098 5.5 5:a4ac28f7:::benchmark_data_smithi032_22800_object74097:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.043782+0000", 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.012718464000000001, 2024-04-18T22:23:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013396399999999999, 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74098 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043782+0000", 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043782+0000", 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043783+0000", 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6700000000000003e-07 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043897+0000", 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011394 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043908+0000", 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1419e-05 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.043914+0000", 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.404e-06 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044382+0000", 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046762600000000001 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044504+0000", 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122438 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044640+0000", 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013542599999999999 2024-04-18T22:23:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74100 5.5 5:aead745a:::benchmark_data_smithi032_22800_object74099:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.044498+0000", 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.012002383, 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012747414, 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74100 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044498+0000", 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044498+0000", 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044499+0000", 2024-04-18T22:23:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.47e-07 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044589+0000", 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0024000000000004e-05 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044600+0000", 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1113e-05 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.044607+0000", 2024-04-18T22:23:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6640000000000001e-06 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056069+0000", 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011462221999999999 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056230+0000", 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000161368 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056385+0000", 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015430500000000001 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74101 5.5 5.93eb7af5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.045270+0000", 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011230449999999999, 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012038241, 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74101 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045270+0000", 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045270+0000", 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045271+0000", 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5600000000000005e-07 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045356+0000", 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5766000000000005e-05 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045364+0000", 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8089999999999993e-06 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045369+0000", 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7010000000000004e-06 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:74099 5.b e34/19 5:dd45f45b:::benchmark_data_smithi032_22800_object74098:head v 34'4649, mlcod=34'4645)", 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.045256+0000", 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011244475, 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012105367000000001, 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045256+0000", 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045256+0000", 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045257+0000", 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9100000000000003e-07 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045362+0000", 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105398 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045381+0000", 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8618e-05 2024-04-18T22:23:28.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.045388+0000", 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5569999999999998e-06 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047436+0000", 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020482270000000001 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047454+0000", 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7207999999999999e-05 2024-04-18T22:23:28.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74102 5.7 5:eca569fb:::benchmark_data_smithi032_22800_object74101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.046543+0000", 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0099576260000000007, 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010875064, 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74102 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.046543+0000", 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.046543+0000", 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.046543+0000", 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4900000000000001e-07 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.046571+0000", 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.758e-05 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.046577+0000", 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4919999999999998e-06 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.046582+0000", 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1909999999999999e-06 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056095+0000", 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0095123859999999994 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056262+0000", 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000167076 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056387+0000", 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125332 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:74090 5.5 e34/19)", 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.047032+0000", 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0094682359999999997, 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010447155, 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.120 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047032+0000", 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047032+0000", 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047033+0000", 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5899999999999996e-07 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047037+0000", 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8249999999999998e-06 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047047+0000", 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0361e-05 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047053+0000", 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1670000000000004e-06 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:74104 5.c 5:371926f4:::benchmark_data_smithi032_22800_object74103:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T22:23:28.123 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:28.047410+0000", 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0090903300000000006, 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010115406, 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 74104 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047410+0000", 2024-04-18T22:23:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047410+0000", 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047411+0000", 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7899999999999997e-07 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047508+0000", 2024-04-18T22:23:28.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7584999999999995e-05 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047518+0000", 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5410000000000006e-06 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.047525+0000", 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0029999999999997e-06 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056092+0000", 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0085672679999999994 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056234+0000", 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014181299999999999 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:28.056378+0000", 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144294 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.128 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:28.128 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 18 2024-04-18T22:23:28.128 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:28.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:28.130 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.126+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.192 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 32 16 74377 74361 145.221 122.875 0.0086543 0.00688085 2024-04-18T22:23:28.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:28.227+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:28.319 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:28.319 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:28.320 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:28.320 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:28.320 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:28.331 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.330+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.340 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:28.432 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.430+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.532 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.530+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.421057421999997, 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:28.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:28.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:28.543 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420924929999998, 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:28.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:28.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420853344000001, 2024-04-18T22:23:28.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:28.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:28.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.555 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420725091000001, 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:28.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:28.561 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420615763999997, 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:28.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:28.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:28.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:28.567 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420359427000001, 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:28.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:28.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:28.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:28.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420271941000003, 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.420193955000002, 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:28.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:28.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:28.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.737906873, 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:28.583 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:28.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:28.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:28.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.736250494, 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.594 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.736125777000002, 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.734346119000001, 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:28.600 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.733863448000001, 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:28.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:28.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.733624373000001, 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:28.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.733503601999999, 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:28.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.730614190000001, 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:28.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:28.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:28.628 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.730492371, 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:28.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:28.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.633 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.730381921999999, 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.639 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.730312454, 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:28.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:28.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.730255313000001, 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:28.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:28.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:28.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:28.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:28.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:28.650 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:28.651 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:28.652 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:28.651+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:28.652 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:28.752 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.750+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75517 5.5 e34/19 5:a790fd28:::benchmark_data_smithi032_22800_object75516:head v 34'4672, mlcod=34'4669)", 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.799715+0000", 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018479869999999999, 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001870752, 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799715+0000", 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799715+0000", 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799716+0000", 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799803+0000", 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6918999999999999e-05 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799812+0000", 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7930000000000003e-06 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799818+0000", 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799868+0000", 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9835e-05 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799888+0000", 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9749999999999999e-05 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75518 5.7 e34/23 5:e63c07cb:::benchmark_data_smithi032_22800_object75517:head v 34'4747, mlcod=34'4744)", 2024-04-18T22:23:28.804 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.799838+0000", 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0017252630000000001, 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001797802, 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799838+0000", 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799838+0000", 2024-04-18T22:23:28.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799839+0000", 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1199999999999998e-07 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799930+0000", 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1292999999999995e-05 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799940+0000", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0148e-05 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799946+0000", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3470000000000004e-06 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799986+0000", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9990999999999997e-05 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800003+0000", 2024-04-18T22:23:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6657e-05 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75520 5.d e34/19 5:b44cda2f:::benchmark_data_smithi032_22800_object75519:head v 34'4687, mlcod=34'4685)", 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.800267+0000", 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0012960249999999999, 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001399958, 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800267+0000", 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800267+0000", 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800268+0000", 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4600000000000005e-07 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800315+0000", 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7063e-05 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800321+0000", 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2110000000000003e-06 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800337+0000", 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6116e-05 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800383+0000", 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5383999999999998e-05 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.800402+0000", 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9774000000000001e-05 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75519 5.9 e34/23)", 2024-04-18T22:23:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.801435+0000", 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00012832499999999999, 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026054499999999999, 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801435+0000", 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801435+0000", 2024-04-18T22:23:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801436+0000", 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.63e-07 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801524+0000", 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8174e-05 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801532+0000", 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9540000000000004e-06 2024-04-18T22:23:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801537+0000", 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5419999999999996e-06 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75510 5.9 e34/23 5:9e69ef47:::benchmark_data_smithi032_22800_object75509:head v 34'4724, mlcod=34'4722)", 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.793638+0000", 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0079256320000000002, 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080855570000000002, 2024-04-18T22:23:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793638+0000", 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793638+0000", 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793639+0000", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5300000000000003e-07 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793749+0000", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011020599999999999 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793761+0000", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2408999999999999e-05 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793769+0000", 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9400000000000002e-06 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793823+0000", 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4209999999999998e-05 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.793844+0000", 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0775999999999999e-05 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75513 5.1 e34/23 5:82b63f80:::benchmark_data_smithi032_22800_object75512:head v 34'4661, mlcod=34'4659)", 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.796486+0000", 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0050773779999999996, 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052680970000000002, 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796486+0000", 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796486+0000", 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796486+0000", 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.75e-07 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796538+0000", 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1924e-05 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796560+0000", 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1882000000000002e-05 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796567+0000", 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2600000000000002e-06 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796640+0000", 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3714000000000006e-05 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.796669+0000", 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9003e-05 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75515 5.7 e34/23 5:ed1a1c7f:::benchmark_data_smithi032_22800_object75514:head v 34'4746, mlcod=34'4744)", 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.797534+0000", 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0040295549999999998, 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042674100000000001, 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797534+0000", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797534+0000", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797534+0000", 2024-04-18T22:23:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5000000000000003e-07 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797627+0000", 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2493999999999999e-05 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797637+0000", 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0244999999999999e-05 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797642+0000", 2024-04-18T22:23:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5920000000000002e-06 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797693+0000", 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1399e-05 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.797711+0000", 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8303000000000001e-05 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801770+0000", 2024-04-18T22:23:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040581239999999998 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.801801+0000", 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1702999999999998e-05 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:75516 5.9 e34/23 5:98be8a71:::benchmark_data_smithi032_22800_object75515:head v 34'4725, mlcod=34'4722)", 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:28.799528+0000", 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002035556, 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002361045, 2024-04-18T22:23:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799528+0000", 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799528+0000", 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799528+0000", 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2200000000000004e-07 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799589+0000", 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0427999999999998e-05 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799598+0000", 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3880000000000006e-06 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799603+0000", 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6029999999999998e-06 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799639+0000", 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5777e-05 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:28.799651+0000", 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1902999999999999e-05 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 8 2024-04-18T22:23:28.829 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:28.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:28.853 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.850+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.950 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:23:28.954 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:28.954+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:28.974 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:28.975 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:28.975 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:28.975 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:28.975 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:28.995 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:29.055 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:29.054+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:29.135 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:29.135 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:29.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:29.135 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019825996999998, 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:29.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:29.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019757399, 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.141 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:29.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:29.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:29.146 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019661552999999, 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:29.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019604671000003, 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:29.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:29.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.156 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019544998999997, 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019487036999998, 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:29.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:29.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:29.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019267345000003, 2024-04-18T22:23:29.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019210416, 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:29.177 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019114530000003, 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.019052201999997, 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:29.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:29.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:29.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:29.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.018979723999998, 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.191 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:29.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:29.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.018827256000002, 2024-04-18T22:23:29.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:29.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:29.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:29.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.201 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.018748533, 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.206 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.017227677000001, 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:29.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:29.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:29.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:29.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.017129324999999, 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.335407371999999, 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:29.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:29.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:29.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:29.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.334753159000002, 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.334646463999999, 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.332780554999999, 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.228 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:29.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:29.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:29.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.332115232, 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:29.232 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:29.236 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:29.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:29.238 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 33 16 76294 76278 144.45 119.812 0.00649314 0.00691804 2024-04-18T22:23:29.238 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:29.238+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:29.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:29.338 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:29.338+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:76659 5.0 e35/19 5:03c3bd8c:::benchmark_data_smithi032_22800_object76658:head v 35'4821, mlcod=35'4819)", 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:29.383222+0000", 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0038184989999999999, 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038338959999999998, 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:29.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383222+0000", 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383222+0000", 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383223+0000", 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7e-07 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.390 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383258+0000", 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5394e-05 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383267+0000", 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9109999999999999e-06 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383273+0000", 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6640000000000003e-06 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383321+0000", 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7817e-05 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.383339+0000", 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8808999999999999e-05 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:76660 5.3 5:c5dedbb3:::benchmark_data_smithi032_22800_object76659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:29.384345+0000", 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002695968, 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002750457, 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 76660 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384345+0000", 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384345+0000", 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384346+0000", 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.186e-06 2024-04-18T22:23:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384426+0000", 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9455000000000004e-05 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384441+0000", 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5237e-05 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384449+0000", 2024-04-18T22:23:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2099999999999993e-06 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384524+0000", 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5159000000000003e-05 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384671+0000", 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014701699999999999 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.384813+0000", 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014196900000000001 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.386834+0000", 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020209569999999999 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:76661 5.8 e35/19 5:1cb304ba:::benchmark_data_smithi032_22800_object76660:head v 35'4701, mlcod=35'4699)", 2024-04-18T22:23:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:29.385124+0000", 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0019165499999999999, 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020136059999999998, 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385124+0000", 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385124+0000", 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385125+0000", 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0100000000000005e-07 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385182+0000", 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6718999999999997e-05 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385193+0000", 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1600000000000001e-05 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385200+0000", 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5169999999999996e-06 2024-04-18T22:23:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385255+0000", 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.499e-05 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385281+0000", 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6332000000000001e-05 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:76657 5.4 e35/19 5:274df90e:::benchmark_data_smithi032_22800_object76656:head v 35'4911, mlcod=35'4908)", 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:29.385220+0000", 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0018212320000000001, 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019525090000000001, 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385220+0000", 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385220+0000", 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385220+0000", 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4999999999999998e-07 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385339+0000", 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118764 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385348+0000", 2024-04-18T22:23:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9500000000000007e-06 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385353+0000", 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5899999999999998e-06 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385395+0000", 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2001e-05 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:29.385414+0000", 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8630999999999998e-05 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 4 2024-04-18T22:23:29.406 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:29.407 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:29.408 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:29.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:29.420 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-04-18T22:23:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T22:23:29.439 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:29.440 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:29.440 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:29.439+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:29.540 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:29.539+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:29.606 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:29.606 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:29.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:29.673 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:29.674+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:29.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:29.739+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:29.774 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:29.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:29.843 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:29.842+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:29.943 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:29.944 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:29.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:29.943+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:29.960 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:29.988 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T22:23:30.046 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:30.046+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:30.099 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-04-18T22:23:30.121 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:30.121 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:30.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:30.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:30.183+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:30.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 34 16 77732 77716 142.845 89.875 0.00962297 0.00694655 2024-04-18T22:23:30.247 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:30.246+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:30.258 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:30.259 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:30.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:30.347 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:30.346+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:30.393 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:30.393 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:30.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:30.449 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:30.450+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:30.549 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:30.547+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:30.565 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:30.565 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:30.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:30.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:30.651+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:30.686 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:30.687 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:30.699 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:30.752 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:30.751+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:30.832 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:30.832 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:30.846 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:30.852 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:30.850+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:30.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:30.951+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:30.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:23:31.024 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:31.024 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:31.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:31.054 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:31.054+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:31.155 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:31.154+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:31.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 35 16 78064 78048 139.356 20.75 0.00899169 0.00717085 2024-04-18T22:23:31.239 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:31.239 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:31.256 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:31.255+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:31.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:31.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:31.355+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:31.434 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:31.435 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:31.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:31.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:31.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:31.459+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:31.559 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:31.559+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:31.614 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:31.615 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:31.623 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:31.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:31.642 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:31.645 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:31.646 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:31.681 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:31.682+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:31.762 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:31.762+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:31.800 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:31.800 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.683383861000003, 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:31.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:31.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.683251368999997, 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:31.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:31.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:31.814 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.683179783, 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:31.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:31.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.820 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.68305153, 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:31.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:31.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:31.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:31.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.826 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.682942203000003, 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:31.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:31.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:31.832 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.682685866, 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:31.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:31.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:31.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.682598380000002, 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:31.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 59.682520394000001, 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:31.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.000233311999999, 2024-04-18T22:23:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:31.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:31.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.998576932999999, 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.855 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:31.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:31.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.998452216, 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.862 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.996672558, 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:31.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.996189887, 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:31.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:31.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.995950812, 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.879 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:31.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:31.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.995830041000001, 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:31.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:31.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.992940629, 2024-04-18T22:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:31.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:31.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:31.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.992818809999999, 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:31.902 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.992708360999998, 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.903 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:31.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:31.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.992638892999999, 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:31.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:31.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:31.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 25.992581752, 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:31.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:31.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:31.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:31.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:31.920 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:31.921 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:31.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:31.923 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:31.922+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:31.976 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:23:32.023 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:32.022+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:32.074 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:32.074 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:32.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.074 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:79777 5.9 5:95149644:::benchmark_data_smithi032_22800_object79776:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.075 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:32.073975+0000", 2024-04-18T22:23:32.075 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0011489040000000001, 2024-04-18T22:23:32.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011772550000000001, 2024-04-18T22:23:32.075 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.075 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.075 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:32.076 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:32.076 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:32.076 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.076 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 79777 2024-04-18T22:23:32.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.077 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.073975+0000", 2024-04-18T22:23:32.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.073975+0000", 2024-04-18T22:23:32.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.073976+0000", 2024-04-18T22:23:32.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6099999999999999e-07 2024-04-18T22:23:32.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074034+0000", 2024-04-18T22:23:32.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7578999999999999e-05 2024-04-18T22:23:32.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074043+0000", 2024-04-18T22:23:32.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.747e-06 2024-04-18T22:23:32.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074051+0000", 2024-04-18T22:23:32.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7780000000000007e-06 2024-04-18T22:23:32.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074092+0000", 2024-04-18T22:23:32.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0646999999999999e-05 2024-04-18T22:23:32.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074178+0000", 2024-04-18T22:23:32.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6636000000000002e-05 2024-04-18T22:23:32.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.084 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.085 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:79775 5.5 e38/19 5:a830f128:::benchmark_data_smithi032_22800_object79774:head v 38'4962, mlcod=38'4960)", 2024-04-18T22:23:32.085 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:32.074364+0000", 2024-04-18T22:23:32.085 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000759639, 2024-04-18T22:23:32.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00085750300000000002, 2024-04-18T22:23:32.085 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.086 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.086 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:32.086 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074364+0000", 2024-04-18T22:23:32.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074364+0000", 2024-04-18T22:23:32.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074365+0000", 2024-04-18T22:23:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0500000000000004e-07 2024-04-18T22:23:32.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074451+0000", 2024-04-18T22:23:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6582999999999994e-05 2024-04-18T22:23:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074462+0000", 2024-04-18T22:23:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0344e-05 2024-04-18T22:23:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074468+0000", 2024-04-18T22:23:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.942e-06 2024-04-18T22:23:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074505+0000", 2024-04-18T22:23:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7292000000000001e-05 2024-04-18T22:23:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:32.074517+0000", 2024-04-18T22:23:32.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2315e-05 2024-04-18T22:23:32.093 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.093 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.093 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.093 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.094 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:32.094 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 2 2024-04-18T22:23:32.094 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:32.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:32.147 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:32.147+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:32.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 36 16 80047 80031 138.928 123.938 0.0075604 0.0071929 2024-04-18T22:23:32.224 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:32.222+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:32.243 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:32.243 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:32.243 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:32.243 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:32.243 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:32.264 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:32.355 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:32.355+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:32.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:32.427+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:32.461 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:32.461 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:32.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:32.461 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:32.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.461 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345738773999997, 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.462 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:32.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:32.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:32.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345670175999999, 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.467 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.472 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345574329999998, 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:32.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.477 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345517448000003, 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:32.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345457776000003, 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:32.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:32.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.488 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345399813999997, 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.489 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:32.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:32.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:32.493 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345180122000002, 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345123192999999, 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.499 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:32.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.504 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.345027307000002, 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:32.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:32.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.344964978999997, 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.510 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:32.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:32.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.344892500999997, 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:32.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.344740033000001, 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.521 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:32.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.526 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.344661309999999, 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:32.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:32.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.531 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.343140454, 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:32.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:32.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 60.343042101999998, 2024-04-18T22:23:32.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:32.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:32.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 26.661320149000002, 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:32.541 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:32.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:32.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 26.660665936000001, 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.549 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 26.660559241000001, 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 26.658693331999999, 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.554 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:32.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:32.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:32.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 26.658028008999999, 2024-04-18T22:23:32.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:32.562 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:32.564 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:32.562+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:32.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:32.664 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:32.663+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:32.723 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81178 5.c 5:34783412:::benchmark_data_smithi032_22800_object81177:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.724318+0000", 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000380081, 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00040435500000000001, 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81178 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724318+0000", 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724318+0000", 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724319+0000", 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0419999999999999e-06 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724351+0000", 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2387e-05 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724363+0000", 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1167e-05 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724370+0000", 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4529999999999996e-06 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724410+0000", 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9536999999999998e-05 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724493+0000", 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3261000000000001e-05 2024-04-18T22:23:32.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724583+0000", 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9838000000000003e-05 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:81169 5.1 e38/23 ondisk, result = 0)", 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.724667+0000", 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.1445999999999997e-05, 2024-04-18T22:23:32.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012964899999999999, 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724667+0000", 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724667+0000", 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724667+0000", 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1099999999999996e-07 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724671+0000", 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7079999999999999e-06 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724680+0000", 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6670000000000008e-06 2024-04-18T22:23:32.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724687+0000", 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.263e-06 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724736+0000", 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9329000000000002e-05 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724743+0000", 2024-04-18T22:23:32.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.126e-06 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81179 5.4 5:2da62d1f:::benchmark_data_smithi032_22800_object81178:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.724666+0000", 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.1702000000000002e-05, 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000194446, 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.732 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81179 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724666+0000", 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724666+0000", 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724667+0000", 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5199999999999996e-07 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724689+0000", 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1807999999999999e-05 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724696+0000", 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2219999999999996e-06 2024-04-18T22:23:32.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724701+0000", 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6970000000000001e-06 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724744+0000", 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3439000000000002e-05 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724862+0000", 2024-04-18T22:23:32.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118047 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81180 5.4 5.74746294 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.724800+0000", 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998975, 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011976000000000001, 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.736 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81180 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724800+0000", 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724800+0000", 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724801+0000", 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2900000000000004e-07 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724873+0000", 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1693999999999995e-05 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724879+0000", 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5780000000000004e-06 2024-04-18T22:23:32.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724885+0000", 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.468e-06 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81170 5.8 5:1021a920:::benchmark_data_smithi032_22800_object81169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.719224+0000", 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0054742419999999998, 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005751529, 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81170 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719224+0000", 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719224+0000", 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719225+0000", 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0109999999999999e-06 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719291+0000", 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5813000000000002e-05 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719303+0000", 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2683e-05 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719310+0000", 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1790000000000002e-06 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719373+0000", 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2271000000000001e-05 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719511+0000", 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013864199999999999 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81171 5.7 5:ee06ad9a:::benchmark_data_smithi032_22800_object81170:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.719704+0000", 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0049938279999999996, 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053388960000000001, 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81171 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719704+0000", 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719704+0000", 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719705+0000", 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0100000000000004e-07 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719829+0000", 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012368500000000001 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719839+0000", 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0443e-05 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.719847+0000", 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5809999999999999e-06 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721285+0000", 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014380179999999999 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721376+0000", 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1441999999999996e-05 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721466+0000", 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0012999999999994e-05 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724550+0000", 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030834550000000001 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.747 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81172 5.4 5:28aa437b:::benchmark_data_smithi032_22800_object81171:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.721658+0000", 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030402900000000002, 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003461062, 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81172 2024-04-18T22:23:32.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721658+0000", 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721658+0000", 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721658+0000", 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.92e-07 2024-04-18T22:23:32.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721751+0000", 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2850999999999996e-05 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721761+0000", 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0161e-05 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721771+0000", 2024-04-18T22:23:32.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5429999999999999e-06 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721844+0000", 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2574e-05 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721961+0000", 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011745599999999999 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81174 5.8 5:1168444c:::benchmark_data_smithi032_22800_object81173:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.721842+0000", 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0028563870000000002, 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033409189999999999, 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81174 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721842+0000", 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721842+0000", 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721843+0000", 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6700000000000002e-07 2024-04-18T22:23:32.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721931+0000", 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8274000000000002e-05 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721942+0000", 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0951999999999999e-05 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721950+0000", 2024-04-18T22:23:32.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7179999999999996e-06 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722007+0000", 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7394e-05 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722121+0000", 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011411 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81175 5.5 5:a7d68f31:::benchmark_data_smithi032_22800_object81174:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.721970+0000", 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0027279370000000002, 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032736200000000001, 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81175 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.756 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721970+0000", 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721970+0000", 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.721971+0000", 2024-04-18T22:23:32.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1800000000000005e-07 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722010+0000", 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9388000000000003e-05 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722019+0000", 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7099999999999996e-06 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722025+0000", 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4320000000000004e-06 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722082+0000", 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6607999999999997e-05 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722203+0000", 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121377 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722358+0000", 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000154253 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724599+0000", 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022415949999999999 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81176 5.8 5:13ba513a:::benchmark_data_smithi032_22800_object81175:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.760 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.722043+0000", 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002655452, 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032727949999999998, 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81176 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722043+0000", 2024-04-18T22:23:32.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722043+0000", 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722043+0000", 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7700000000000004e-07 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722081+0000", 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7642000000000003e-05 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722090+0000", 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7229999999999999e-06 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722097+0000", 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4110000000000001e-06 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.723890+0000", 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001792727 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.724014+0000", 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012480799999999999 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:81177 5.e 5:7e4f5177:::benchmark_data_smithi032_22800_object81176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:32.722107+0000", 2024-04-18T22:23:32.764 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025908179999999999, 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003273111, 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 81177 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722107+0000", 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722107+0000", 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722108+0000", 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5300000000000004e-07 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722145+0000", 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7431000000000001e-05 2024-04-18T22:23:32.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722154+0000", 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8550000000000008e-06 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722160+0000", 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0059999999999998e-06 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722223+0000", 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3202999999999998e-05 2024-04-18T22:23:32.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.722349+0000", 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012619500000000001 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:32.725389+0000", 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030397330000000002 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:32.768 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 11 2024-04-18T22:23:32.769 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:32.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:32.771 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:32.770+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:32.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:32.871+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:32.959 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:32.959 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:32.960 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:32.960 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:32.960 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:32.972 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:32.971+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:32.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:33.073 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:33.071+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.022554911, 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:33.139 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:33.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:33.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:33.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.022422419000002, 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:33.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:33.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:33.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.022350832999997, 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:33.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.022222579999998, 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:33.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:33.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:33.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.022113253000001, 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:33.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:33.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:33.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.021856915999997, 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:33.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.021769429999999, 2024-04-18T22:23:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:33.178 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.021691443999998, 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.183 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.339404362, 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:33.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.337747983, 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:33.189 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:33.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.337623266000001, 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:33.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.335843608000001, 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:33.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.206 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.335360937000001, 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.335121862000001, 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:33.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:33.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:33.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.335001090999999, 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.332111679, 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.33198986, 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.331879410999999, 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.331809943, 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.331752802, 2024-04-18T22:23:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:33.252 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:33.253 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 37 15 82126 82111 138.686 130 0.0101072 0.00720574 2024-04-18T22:23:33.254 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:33.254 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.250+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:33.354 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.354+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:33.407 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82559 5.7 e38/23 5:eeba9c78:::benchmark_data_smithi032_22800_object82558:head v 38'5193, mlcod=38'5191)", 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.402189+0000", 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0060410350000000002, 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060659140000000004, 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402189+0000", 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402189+0000", 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402189+0000", 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8599999999999998e-07 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.409 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402261+0000", 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1884000000000005e-05 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402270+0000", 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.67e-06 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402275+0000", 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4280000000000004e-06 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402348+0000", 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3129999999999999e-05 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402382+0000", 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3209e-05 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.411 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82561 5.5 e38/19 5:a3d497ed:::benchmark_data_smithi032_22800_object82560:head v 38'5115, mlcod=38'5113)", 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.402289+0000", 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0059404480000000001, 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0060267910000000001, 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402289+0000", 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402289+0000", 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402289+0000", 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4999999999999999e-07 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402323+0000", 2024-04-18T22:23:33.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3816000000000003e-05 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402328+0000", 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3950000000000003e-06 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402331+0000", 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8839999999999998e-06 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402372+0000", 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1664e-05 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402390+0000", 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8162999999999999e-05 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82563 5.7 e38/23 5:efee2d6a:::benchmark_data_smithi032_22800_object82562:head v 38'5194, mlcod=38'5191)", 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.402607+0000", 2024-04-18T22:23:33.415 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0056223310000000004, 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0057752919999999996, 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402607+0000", 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402607+0000", 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402608+0000", 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3799999999999998e-07 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402674+0000", 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6208e-05 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402680+0000", 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0549999999999996e-06 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.402684+0000", 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9349999999999996e-06 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404522+0000", 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001838395 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404534+0000", 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2099999999999999e-05 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82567 5.5 e38/19 5:a9e46e84:::benchmark_data_smithi032_22800_object82566:head v 38'5116, mlcod=38'5113)", 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.403424+0000", 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0048057029999999997, 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005012987, 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.419 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.403424+0000", 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.403424+0000", 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.403424+0000", 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0699999999999998e-07 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.403501+0000", 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6377000000000004e-05 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.403507+0000", 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0039999999999996e-06 2024-04-18T22:23:33.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.403510+0000", 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5329999999999999e-06 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404453+0000", 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00094294599999999995 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404474+0000", 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0514e-05 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82568 5.2 e38/23 5:4d526795:::benchmark_data_smithi032_22800_object82567:head v 38'5158, mlcod=38'5155)", 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.404792+0000", 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0034372420000000001, 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036998460000000001, 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.423 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404792+0000", 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404792+0000", 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404793+0000", 2024-04-18T22:23:33.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7400000000000003e-07 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404919+0000", 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000125668 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404927+0000", 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6670000000000002e-06 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404932+0000", 2024-04-18T22:23:33.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8369999999999996e-06 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404982+0000", 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0281999999999997e-05 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405004+0000", 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1863000000000001e-05 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:82569 5.9 5:99170f57:::benchmark_data_smithi032_22800_object82568:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.404886+0000", 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033436709999999999, 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036660659999999999, 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 82569 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.427 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404886+0000", 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404886+0000", 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.404887+0000", 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0299999999999998e-07 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405005+0000", 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011779500000000001 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405017+0000", 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2203e-05 2024-04-18T22:23:33.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405025+0000", 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8269999999999998e-06 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405080+0000", 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4991999999999998e-05 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405226+0000", 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000146563 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.431 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:82570 5.9 5:96199994:::benchmark_data_smithi032_22800_object82569:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.405042+0000", 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003187685, 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035781239999999998, 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 82570 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405042+0000", 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405042+0000", 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405043+0000", 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3e-07 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405071+0000", 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8496e-05 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405079+0000", 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9470000000000003e-06 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.405085+0000", 2024-04-18T22:23:33.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6019999999999999e-06 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407402+0000", 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023171759999999998 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407568+0000", 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016563099999999999 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82573 5.2 e38/23 5:43ba6414:::benchmark_data_smithi032_22800_object82572:head v 38'5159, mlcod=38'5156)", 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.407397+0000", 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00083257399999999999, 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001283129, 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407397+0000", 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407397+0000", 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407398+0000", 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3000000000000001e-07 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407456+0000", 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8378000000000002e-05 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407469+0000", 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3032e-05 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407474+0000", 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.242e-06 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407524+0000", 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9917999999999999e-05 2024-04-18T22:23:33.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407543+0000", 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9253e-05 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:82572 5.6 e38/19 5:665ccb9b:::benchmark_data_smithi032_22800_object82571:head v 38'5125, mlcod=38'5122)", 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:33.407487+0000", 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00074288100000000003, 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001249249, 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.439 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407487+0000", 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407487+0000", 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407487+0000", 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5699999999999998e-07 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407559+0000", 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1978999999999996e-05 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407568+0000", 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2960000000000004e-06 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407574+0000", 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7350000000000001e-06 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407607+0000", 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3176000000000002e-05 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:33.407618+0000", 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1062e-05 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.442 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.443 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.443 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:33.443 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:23:33.443 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:33.444 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:33.455 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.454+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:33.556 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.554+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:33.592 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:33.592 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:33.592 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:33.592 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:33.592 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:33.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:33.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:33.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:33.663+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:33.757 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.754+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.651481668000002, 2024-04-18T22:23:33.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:33.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.772 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.651413069999997, 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:33.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:33.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:33.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.651317224000003, 2024-04-18T22:23:33.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:33.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:33.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:33.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.651260342, 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:33.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:33.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:33.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:33.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:33.789 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.651200670000001, 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:33.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:33.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:33.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:33.795 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.651142708000002, 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.800 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650923016, 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:33.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:33.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:33.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650866086999997, 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.806 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:33.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:33.811 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650770201, 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:33.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:33.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:33.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:33.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650707873000002, 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.817 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:33.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:33.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:33.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:33.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650635395000002, 2024-04-18T22:23:33.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:33.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:33.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650482926999999, 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:33.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:33.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.832 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.650404203999997, 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:33.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.648883347999998, 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.838 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:33.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:33.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:33.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.648784996000003, 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.843 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:33.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:33.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:33.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.967063043, 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:33.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:33.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:33.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.966408829999999, 2024-04-18T22:23:33.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:33.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:33.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:33.855 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.966302134999999, 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:33.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:33.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.964436226, 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.963770903, 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:33.868 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:33.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:33.870 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.866+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:33.870 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:33.870 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:33.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:33.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:33.872 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:33.872 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:33.872 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:33.970 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:33.970+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:34.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:23:34.028 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:34.028 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:34.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:34.094 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:34.094+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:34.100 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T22:23:34.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:34.171+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:34.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 38 16 84205 84189 138.454 129.875 0.00594476 0.00721772 2024-04-18T22:23:34.210 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-04-18T22:23:34.240 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:34.240 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:34.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:34.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-18T22:23:34.273 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:34.271+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:34.374 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:34.374+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:34.415 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:34.415 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:34.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:34.483 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:34.483+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:34.576 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:34.574+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:34.580 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:34.581 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:34.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:34.687 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:34.687+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:34.778 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:34.775+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:34.786 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:34.786 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:34.808 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:34.879 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:34.878+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:34.980 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:34.978+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:34.992 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:23:34.997 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:34.997 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:35.020 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:35.081 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.078+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.172 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:35.172 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:35.182 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.182+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:35.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 39 16 86224 86208 138.139 126.188 0.00655743 0.00723409 2024-04-18T22:23:35.286 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.282+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.385 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.382+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.386 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:35.386 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:35.407 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:35.494 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.494+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:35.587+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:35.606 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:35.606 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:35.627 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:35.688 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:35.687+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:35.788 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.786+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.789 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:35.789 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:35.810 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:35.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:35.889 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:35.886+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:35.990 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:35.987+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:36.002 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:36.002 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:36.021 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:36.035 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:36.036 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:36.036 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:36.053 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:36.056 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:36.057 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:36.108 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:36.107+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:36.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:36.191+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:36.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:2024-04-18T22:23:36.195392+0000 min lat: 0.00133878 max lat: 1.06735 avg lat: 0.00726265 2024-04-18T22:23:36.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-18T22:23:36.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 40 16 88089 88073 137.599 116.562 0.010039 0.00726265 2024-04-18T22:23:36.210 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.09501444, 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.211 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:36.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:36.217 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094945842000001, 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:36.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:36.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094849995999994, 2024-04-18T22:23:36.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:36.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:36.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:36.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:36.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:36.229 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094793113999998, 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:36.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:36.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094733442000006, 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:36.235 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:36.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094675480000006, 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:36.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094455788000005, 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:36.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:36.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.251 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094398858999995, 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:36.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:36.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:36.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:36.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094302972999998, 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094240644999999, 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094168167000007, 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.094015698999996, 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.093936975999995, 2024-04-18T22:23:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.092416119999996, 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 64.092317768000001, 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.410595815000001, 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.409941602, 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.409834907, 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.407968998000001, 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.407303675000001, 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:36.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:36.316 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:36.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:36.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:36.315+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:36.418 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:36.418+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:36.467 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:36.467 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88588 5.e 5:74b4376d:::benchmark_data_smithi032_22800_object88587:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.461763+0000", 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0060564399999999997, 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060816100000000003, 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88588 2024-04-18T22:23:36.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461763+0000", 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461763+0000", 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461764+0000", 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4799999999999999e-07 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461786+0000", 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2374e-05 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461791+0000", 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5290000000000002e-06 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461794+0000", 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5810000000000001e-06 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462428+0000", 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00063389999999999996 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462542+0000", 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011362 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.471 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88589 5.e 5:70a734fc:::benchmark_data_smithi032_22800_object88588:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.461801+0000", 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0060184770000000004, 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0061174879999999999, 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88589 2024-04-18T22:23:36.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461801+0000", 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461801+0000", 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461802+0000", 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8299999999999998e-07 2024-04-18T22:23:36.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461825+0000", 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3615999999999999e-05 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461830+0000", 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1100000000000002e-06 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461834+0000", 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1879999999999998e-06 2024-04-18T22:23:36.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464617+0000", 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027838730000000001 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464752+0000", 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013496 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.475 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88590 5.a 5:5c6ab311:::benchmark_data_smithi032_22800_object88589:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.461839+0000", 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0059810200000000001, 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0061445400000000004, 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88590 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461839+0000", 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461839+0000", 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461839+0000", 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.58e-07 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461860+0000", 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1178e-05 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461864+0000", 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0559999999999998e-06 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.461867+0000", 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.976e-06 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464144+0000", 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022764619999999999 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464299+0000", 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015482000000000001 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88591 5.e 5:71190550:::benchmark_data_smithi032_22800_object88590:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.462137+0000", 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0056833989999999996, 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0059085630000000004, 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88591 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462137+0000", 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462137+0000", 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462137+0000", 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.06e-07 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462191+0000", 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4082999999999997e-05 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462197+0000", 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.195e-06 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.462201+0000", 2024-04-18T22:23:36.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6689999999999999e-06 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467091+0000", 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048900259999999996 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467209+0000", 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011792 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88594 5.2 5:4ef265fc:::benchmark_data_smithi032_22800_object88593:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.464425+0000", 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0033947529999999999, 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036915810000000002, 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88594 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464425+0000", 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464425+0000", 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464426+0000", 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.78e-07 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464456+0000", 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9700999999999999e-05 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464466+0000", 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0212e-05 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464473+0000", 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5460000000000003e-06 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464621+0000", 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014702300000000001 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464752+0000", 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000131908 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464880+0000", 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000127902 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88596 5.2 5:4eb7b63a:::benchmark_data_smithi032_22800_object88595:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.464868+0000", 2024-04-18T22:23:36.488 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0029514160000000001, 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003315859, 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88596 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464868+0000", 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464868+0000", 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.464869+0000", 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1200000000000003e-07 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465003+0000", 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013352899999999999 2024-04-18T22:23:36.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465013+0000", 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.028e-05 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465029+0000", 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6089000000000001e-05 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.466926+0000", 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001896954 2024-04-18T22:23:36.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467054+0000", 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000127596 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467141+0000", 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7579999999999998e-05 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.492 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88597 5.4 5:2b73129b:::benchmark_data_smithi032_22800_object88596:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.465134+0000", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002686258, 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031175479999999999, 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88597 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465134+0000", 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465134+0000", 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465135+0000", 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9299999999999996e-07 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465187+0000", 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2856999999999999e-05 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465197+0000", 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7589999999999992e-06 2024-04-18T22:23:36.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465204+0000", 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1409999999999999e-06 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465254+0000", 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9971e-05 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465399+0000", 2024-04-18T22:23:36.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014497300000000001 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88598 5.2 5.731ec682 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.465223+0000", 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025967199999999998, 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003087195, 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:36.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88598 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465223+0000", 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465223+0000", 2024-04-18T22:23:36.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465224+0000", 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6100000000000001e-07 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465265+0000", 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1111e-05 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465273+0000", 2024-04-18T22:23:36.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1990000000000006e-06 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.465279+0000", 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4049999999999997e-06 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88599 5.c 5:3fd12153:::benchmark_data_smithi032_22800_object88598:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.467377+0000", 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00044304899999999999, 2024-04-18T22:23:36.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00098583999999999998, 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88599 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467377+0000", 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467377+0000", 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467378+0000", 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2999999999999999e-07 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467408+0000", 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9845e-05 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467416+0000", 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4780000000000008e-06 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467422+0000", 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1990000000000003e-06 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467466+0000", 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3653999999999999e-05 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467571+0000", 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105596 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467647+0000", 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5347999999999998e-05 2024-04-18T22:23:36.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88600 5.6 5:680686cb:::benchmark_data_smithi032_22800_object88599:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.467437+0000", 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00038261700000000003, 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00099477900000000011, 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88600 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467437+0000", 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467437+0000", 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467438+0000", 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467463+0000", 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5527000000000001e-05 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467470+0000", 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6579999999999996e-06 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467476+0000", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9139999999999996e-06 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467524+0000", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7633000000000003e-05 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467635+0000", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111272 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467758+0000", 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122674 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88601 5.b 5:de79d896:::benchmark_data_smithi032_22800_object88600:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.467487+0000", 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00033300499999999999, 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001017567, 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88601 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467487+0000", 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467487+0000", 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467487+0000", 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4000000000000002e-07 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467521+0000", 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3507000000000003e-05 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467529+0000", 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6699999999999994e-06 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467535+0000", 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4629999999999998e-06 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467628+0000", 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2739999999999996e-05 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.467748+0000", 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012049600000000001 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88578 5.e 5:789511df:::benchmark_data_smithi032_22800_object88577:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.456581+0000", 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011238919, 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011986597999999999, 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88578 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.456581+0000", 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.456581+0000", 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.456582+0000", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2500000000000005e-07 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.456641+0000", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9323000000000001e-05 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.456650+0000", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1470000000000007e-06 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.456656+0000", 2024-04-18T22:23:36.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1029999999999998e-06 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457542+0000", 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00088550499999999997 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457730+0000", 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018797900000000001 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88580 5.a 5:549a4bec:::benchmark_data_smithi032_22800_object88579:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.516 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.457412+0000", 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.010408311999999999, 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011218521, 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88580 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457412+0000", 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457412+0000", 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457412+0000", 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6300000000000004e-07 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457444+0000", 2024-04-18T22:23:36.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1390999999999998e-05 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457453+0000", 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9299999999999992e-06 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457460+0000", 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0849999999999999e-06 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457707+0000", 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00024685400000000002 2024-04-18T22:23:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.457815+0000", 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000108843 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88583 5.e 5:7e7cf37f:::benchmark_data_smithi032_22800_object88582:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.459769+0000", 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0080504989999999992, 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0089265890000000004, 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88583 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459769+0000", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459769+0000", 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459770+0000", 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1999999999999998e-07 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459869+0000", 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8740000000000007e-05 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459878+0000", 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5690000000000005e-06 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459885+0000", 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8789999999999997e-06 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459942+0000", 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7370000000000001e-05 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.460085+0000", 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014266999999999999 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.523 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:88584 5.a 5:55f37a1a:::benchmark_data_smithi032_22800_object88583:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:36.459901+0000", 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0079184189999999995, 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0088591130000000001, 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 88584 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459901+0000", 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459901+0000", 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459902+0000", 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459939+0000", 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7174e-05 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459947+0000", 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7020000000000001e-06 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.459953+0000", 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1349999999999997e-06 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.460276+0000", 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032299199999999997 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:36.460401+0000", 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012463900000000001 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:36.527 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 15 2024-04-18T22:23:36.528 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:36.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:36.529 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:36.526+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:36.629 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:36.630+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:36.725 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:36.725 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:36.725 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:36.726 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:36.726 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:36.731 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:36.730+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:36.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:36.834 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:36.834+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:36.933 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:36.931+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:36.940 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:36.941 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:36.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:36.941 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:36.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.941 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.941 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:36.942 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.823707561999996, 2024-04-18T22:23:36.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:36.942 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.943 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:36.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:36.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:36.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:36.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:36.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:36.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:36.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:36.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:36.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:36.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:36.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:36.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:36.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:36.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:36.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:36.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:36.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:36.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:36.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:36.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:36.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:36.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:36.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:36.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:36.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:36.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:36.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:36.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:36.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.956 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.823575070000004, 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:36.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.964 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.823503484, 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:36.970 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.823375231, 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:36.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:36.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:36.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:36.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.823265903999996, 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:36.976 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:36.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.823009567, 2024-04-18T22:23:36.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:36.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:36.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.822922081000002, 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.987 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:36.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 64.822844094999994, 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:36.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:36.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:36.996 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.140557012999999, 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:36.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:36.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.138900633999999, 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:37.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:37.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.138775917, 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.008 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:37.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:37.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:37.013 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.136996259, 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:37.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:37.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:37.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:37.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.136513588, 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:37.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:37.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.024 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.136274513, 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:37.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:37.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:37.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:37.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:37.029 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.136153742000001, 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.030 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:37.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.035 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.133264329999999, 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:37.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:37.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.133142510999999, 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:37.041 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:37.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:37.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:37.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.133032062000002, 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.047 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.052 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.132962593999999, 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.132905452999999, 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:37.058 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.064 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.064 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.064 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:37.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:23:37.065 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.063+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:37.066 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:37.166 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.163+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:37.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 41 16 89994 89978 137.147 119.062 0.00699835 0.00728637 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:90020 5.f 5:f0e519e2:::benchmark_data_smithi032_22800_object90019:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.209242+0000", 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002417186, 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024473630000000001, 2024-04-18T22:23:37.211 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 90020 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209242+0000", 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209242+0000", 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209243+0000", 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0750000000000001e-06 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209349+0000", 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106205 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209377+0000", 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8215999999999999e-05 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209387+0000", 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0499999999999997e-06 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209448+0000", 2024-04-18T22:23:37.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1723000000000005e-05 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209557+0000", 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000109008 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.215 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:90022 5.f 5.5c031f5f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.209422+0000", 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0022376029999999999, 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023328709999999998, 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 90022 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209422+0000", 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209422+0000", 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209422+0000", 2024-04-18T22:23:37.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5899999999999996e-07 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209511+0000", 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8437999999999996e-05 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209519+0000", 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8769999999999993e-06 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209523+0000", 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5079999999999999e-06 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:90026 5.d 5:b541b433:::benchmark_data_smithi032_22800_object90025:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.209533+0000", 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0021260699999999999, 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002321624, 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.219 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 90026 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209533+0000", 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209533+0000", 2024-04-18T22:23:37.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209534+0000", 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6000000000000004e-07 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209576+0000", 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2321999999999999e-05 2024-04-18T22:23:37.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209582+0000", 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9429999999999997e-06 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209586+0000", 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7950000000000001e-06 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209636+0000", 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0348e-05 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209770+0000", 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013349499999999999 2024-04-18T22:23:37.223 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:90027 5.f 5.b6119c5f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.209595+0000", 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0020643770000000001, 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023343610000000001, 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.224 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 90027 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209595+0000", 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209595+0000", 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209595+0000", 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7399999999999998e-07 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209624+0000", 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8543000000000001e-05 2024-04-18T22:23:37.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209630+0000", 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3770000000000001e-06 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209635+0000", 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9479999999999999e-06 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.227 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:90019 5.3 e39/19 5:c49112e3:::benchmark_data_smithi032_22800_object90018:head v 39'5758, mlcod=39'5756)", 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.209673+0000", 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0019857999999999998, 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002315101, 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.228 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209673+0000", 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209673+0000", 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209674+0000", 2024-04-18T22:23:37.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1799999999999995e-07 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209702+0000", 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7637999999999999e-05 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209714+0000", 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2877999999999999e-05 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209720+0000", 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3859999999999998e-06 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209793+0000", 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3528000000000005e-05 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209818+0000", 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4703999999999999e-05 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:90023 5.2 e39/23 5:422079bf:::benchmark_data_smithi032_22800_object90022:head v 39'5636, mlcod=39'5634)", 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.209838+0000", 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001820758, 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022098130000000001, 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209838+0000", 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209838+0000", 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209839+0000", 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5000000000000003e-07 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209942+0000", 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103225 2024-04-18T22:23:37.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209978+0000", 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5686999999999999e-05 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.209985+0000", 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1999999999999997e-06 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210046+0000", 2024-04-18T22:23:37.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1274999999999999e-05 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210064+0000", 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7397e-05 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:90029 5.f 5.3bdfdb1f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.210025+0000", 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0016343849999999999, 2024-04-18T22:23:37.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020899069999999998, 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 90029 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210025+0000", 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210025+0000", 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210026+0000", 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8700000000000004e-07 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210053+0000", 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6956000000000001e-05 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210062+0000", 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1949999999999992e-06 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.210068+0000", 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0700000000000003e-06 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:90030 5.1 e39/23 5:8c465740:::benchmark_data_smithi032_22800_object90029:head v 39'5569, mlcod=39'5567)", 2024-04-18T22:23:37.238 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.211310+0000", 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00034901399999999998, 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00086098100000000003, 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211310+0000", 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211310+0000", 2024-04-18T22:23:37.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211311+0000", 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7700000000000004e-07 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211358+0000", 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7735000000000003e-05 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211369+0000", 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0592e-05 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211375+0000", 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6509999999999998e-06 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211450+0000", 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4993999999999994e-05 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211482+0000", 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2317000000000002e-05 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:90031 5.c e39/19 5:3e491c5d:::benchmark_data_smithi032_22800_object90030:head v 39'5602, mlcod=39'5599)", 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:37.211818+0000", 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9998417, 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041974699999999999, 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211818+0000", 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211818+0000", 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211818+0000", 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6199999999999996e-07 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211918+0000", 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9246000000000005e-05 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211928+0000", 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0287e-05 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211935+0000", 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9949999999999999e-06 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211981+0000", 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6456999999999999e-05 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:37.211999+0000", 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7635e-05 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:23:37.245 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:37.247 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:37.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.291+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:37.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.371+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:37.442 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:37.442 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:37.442 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:37.442 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:37.442 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:37.463 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:37.472 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:37.470+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:37.574 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.575+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542715079000004, 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:37.659 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:37.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:37.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542646481000006, 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:37.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:37.670 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542550634999998, 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:37.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542493753000002, 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:37.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.681 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542434080999996, 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542376118999996, 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.687 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:37.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:37.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542156426999995, 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.693 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.695 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542099497999999, 2024-04-18T22:23:37.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:37.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:37.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.542003612000002, 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:37.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:37.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:37.710 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.541941284000004, 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.715 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.541868805999997, 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:37.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.720 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.541716338000001, 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:37.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.541637614999999, 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:37.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.731 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.540116759, 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:37.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:37.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.737 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 65.540018407000005, 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.741 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 31.858296454000001, 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:37.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:37.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:37.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 31.857642241000001, 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.750 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 31.857535546000001, 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:37.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 31.855669636999998, 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.755 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:37.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:37.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.759 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 31.855004313999999, 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:37.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:37.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.763 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.764 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.764 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:37.764 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:37.764 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:37.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:37.766 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:37.762+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:37.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.863+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:37.921 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:37.921 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:37.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91477 5.2 5:4a8dd430:::benchmark_data_smithi032_22800_object91476:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.914813+0000", 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00674094, 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067674379999999998, 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91477 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.922 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914813+0000", 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914813+0000", 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914814+0000", 2024-04-18T22:23:37.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7499999999999998e-07 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914906+0000", 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1952000000000003e-05 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914916+0000", 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0220999999999999e-05 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914922+0000", 2024-04-18T22:23:37.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6030000000000001e-06 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917324+0000", 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024017589999999998 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917440+0000", 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116239 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917547+0000", 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106284 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91479 5.a 5:55536029:::benchmark_data_smithi032_22800_object91478:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.915026+0000", 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0065276689999999998, 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066048929999999997, 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:37.926 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91479 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915026+0000", 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915026+0000", 2024-04-18T22:23:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915026+0000", 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9599999999999999e-07 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915064+0000", 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7616e-05 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915072+0000", 2024-04-18T22:23:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915078+0000", 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.648e-06 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919404+0000", 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043267920000000003 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919530+0000", 2024-04-18T22:23:37.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012560499999999999 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91482 5.c 5:34b80ae3:::benchmark_data_smithi032_22800_object91481:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.915232+0000", 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0063214339999999999, 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064356780000000002, 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91482 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915232+0000", 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915232+0000", 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915233+0000", 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0699999999999996e-07 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915313+0000", 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0054000000000003e-05 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915321+0000", 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4009999999999997e-06 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.915327+0000", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1480000000000003e-06 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916708+0000", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001380336 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916852+0000", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144578 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917023+0000", 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017050799999999999 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919831+0000", 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028081 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:91475 5.2 e39/23 ondisk, result = 0)", 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.916635+0000", 2024-04-18T22:23:37.934 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0049186500000000001, 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050732820000000001, 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916635+0000", 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916635+0000", 2024-04-18T22:23:37.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916636+0000", 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.44e-07 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916639+0000", 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.056e-06 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916648+0000", 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7669999999999998e-06 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.916654+0000", 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5790000000000003e-06 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919408+0000", 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00275363 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919414+0000", 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2840000000000003e-06 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91484 5.5 5:a5463af3:::benchmark_data_smithi032_22800_object91483:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.917580+0000", 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0039739700000000003, 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041607270000000003, 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.938 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91484 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917580+0000", 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917580+0000", 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917581+0000", 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.125e-06 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917664+0000", 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3733000000000006e-05 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917676+0000", 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1073999999999999e-05 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917683+0000", 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8059999999999995e-06 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917741+0000", 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7849999999999997e-05 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917854+0000", 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112837 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918001+0000", 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014669600000000001 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91485 5.3 5:ca64fc50:::benchmark_data_smithi032_22800_object91484:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.917793+0000", 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0037600730000000001, 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039850800000000002, 2024-04-18T22:23:37.942 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91485 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917793+0000", 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917793+0000", 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917794+0000", 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.540000000000001e-07 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917822+0000", 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7971999999999999e-05 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917831+0000", 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4849999999999992e-06 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917838+0000", 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6880000000000002e-06 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.917882+0000", 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4623999999999998e-05 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918004+0000", 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012183199999999999 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918132+0000", 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012817899999999999 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91487 5.3 5.6ccff553 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.946 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.918630+0000", 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0029236370000000002, 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032027940000000001, 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91487 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918630+0000", 2024-04-18T22:23:37.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918630+0000", 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918631+0000", 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5600000000000004e-07 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918705+0000", 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4246999999999996e-05 2024-04-18T22:23:37.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918716+0000", 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.092e-05 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918724+0000", 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.661e-06 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91488 5.2 5.98c2c622 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.918736+0000", 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0028175909999999999, 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031500690000000001, 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91488 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918736+0000", 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918736+0000", 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918736+0000", 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918769+0000", 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2787999999999998e-05 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918777+0000", 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1289999999999996e-06 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918784+0000", 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.489e-06 2024-04-18T22:23:37.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91489 5.a 5.f8c1da2a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.918860+0000", 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002693736, 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030686620000000002, 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91489 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918860+0000", 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918860+0000", 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918860+0000", 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918926+0000", 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5535000000000002e-05 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918935+0000", 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8929999999999994e-06 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.918941+0000", 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9039999999999997e-06 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91490 5.0 5:0e0d756a:::benchmark_data_smithi032_22800_object91489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.919836+0000", 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0017171039999999999, 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021255319999999999, 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91490 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919836+0000", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919836+0000", 2024-04-18T22:23:37.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919837+0000", 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0800000000000003e-07 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919913+0000", 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5964999999999995e-05 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919922+0000", 2024-04-18T22:23:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9490000000000002e-06 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919929+0000", 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0740000000000004e-06 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919976+0000", 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6536000000000003e-05 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920113+0000", 2024-04-18T22:23:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000136892 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91491 5.0 5.c0ffe5a0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.919946+0000", 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0016074010000000001, 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020754699999999998, 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91491 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919946+0000", 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919946+0000", 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919947+0000", 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9299999999999998e-07 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919981+0000", 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3893000000000002e-05 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919989+0000", 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2830000000000001e-06 2024-04-18T22:23:37.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919996+0000", 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9369999999999998e-06 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91492 5.b 5.7c65ac7b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.920101+0000", 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014530190000000001, 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001974104, 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91492 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920101+0000", 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920101+0000", 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920101+0000", 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920166+0000", 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4647999999999995e-05 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920172+0000", 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.263e-06 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920176+0000", 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2710000000000003e-06 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91469 5.a 5:5fc465e4:::benchmark_data_smithi032_22800_object91468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.966 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.910363+0000", 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011190790000000001, 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.01177255, 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91469 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910363+0000", 2024-04-18T22:23:37.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910363+0000", 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910363+0000", 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6400000000000003e-07 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910393+0000", 2024-04-18T22:23:37.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9524000000000001e-05 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910400+0000", 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9979999999999997e-06 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910405+0000", 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7420000000000003e-06 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910457+0000", 2024-04-18T22:23:37.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1174000000000002e-05 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.910586+0000", 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012928000000000001 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:91477 5.2 e39/23)", 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.920511+0000", 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010420920000000001, 2024-04-18T22:23:37.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016833690000000001, 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920511+0000", 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920511+0000", 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920512+0000", 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5600000000000001e-07 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920515+0000", 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9189999999999999e-06 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920522+0000", 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0990000000000001e-06 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920526+0000", 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0110000000000002e-06 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91494 5.5 5:a982834d:::benchmark_data_smithi032_22800_object91493:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.920750+0000", 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00080326499999999995, 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001503767, 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91494 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920750+0000", 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920750+0000", 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920751+0000", 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3200000000000003e-07 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920813+0000", 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.198e-05 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920827+0000", 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3573e-05 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920836+0000", 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1050000000000001e-06 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920873+0000", 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7428000000000001e-05 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.920955+0000", 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2163999999999995e-05 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921039+0000", 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3467000000000006e-05 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:91485 5.3 e39/19)", 2024-04-18T22:23:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.921298+0000", 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00025557200000000003, 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010323280000000001, 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921298+0000", 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921298+0000", 2024-04-18T22:23:37.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921299+0000", 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2900000000000004e-07 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921304+0000", 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0660000000000003e-06 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921315+0000", 2024-04-18T22:23:37.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1216000000000001e-05 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921326+0000", 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1285e-05 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91495 5.c 5:30ca9af6:::benchmark_data_smithi032_22800_object91494:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.921924+0000", 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.99963, 2024-04-18T22:23:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00046068500000000001, 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91495 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.984 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921924+0000", 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921924+0000", 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.921924+0000", 2024-04-18T22:23:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1100000000000005e-07 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.922021+0000", 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6910000000000006e-05 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.922035+0000", 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3953e-05 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.922042+0000", 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1740000000000003e-06 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.922091+0000", 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8322000000000003e-05 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.922193+0000", 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010244 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.922302+0000", 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000108492 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:91475 5.2 5:435b7caa:::benchmark_data_smithi032_22800_object91474:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:37.913751+0000", 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.007803028, 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0087143810000000002, 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 91475 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913751+0000", 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913751+0000", 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913751+0000", 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5000000000000001e-07 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913889+0000", 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013718399999999999 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913899+0000", 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0451e-05 2024-04-18T22:23:37.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913907+0000", 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4869999999999996e-06 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.913954+0000", 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7144999999999997e-05 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914070+0000", 2024-04-18T22:23:37.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116185 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.914199+0000", 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000128854 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:37.919419+0000", 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052205000000000003 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:37.992 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 18 2024-04-18T22:23:37.993 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:37.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:37.995 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:37.991+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:38.014 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:23:38.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:38.094 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:38.091+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:38.152 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:38.152 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:38.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:38.193 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 42 16 92036 92020 136.92 127.625 0.00922019 0.00729867 2024-04-18T22:23:38.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:38.195+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:38.212 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T22:23:38.221 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:38.222 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:38.222 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:38.234 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:38.235 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:38.238 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:38.238 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:38.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:38.295+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:38.322 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2024-04-18T22:23:38.340 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:38.341 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:38.364 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:38.397 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:38.394+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:38.424 INFO:teuthology.orchestra.console.smithi096:Power off completed 2024-04-18T22:23:38.498 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:38.498+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:38.514 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:38.514 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:38.536 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:38.600 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:38.599+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:38.688 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:38.688 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:38.701 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:38.698+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:38.711 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:38.801 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:38.799+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:38.902 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:38.902+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:38.908 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:38.908 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:38.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:39.007 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:39.006+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:39.078 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:39.078 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:39.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:39.142 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:39.142+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:39.194 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 43 16 94121 94105 136.766 130.312 0.00713734 0.00730694 2024-04-18T22:23:39.211 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:39.210+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:39.299 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:39.299 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:39.311 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:39.310+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:39.321 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:39.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:39.411+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:39.513 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:39.510+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:39.523 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:39.523 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:39.543 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:39.613 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:39.611+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:39.695 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:39.695 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:39.713 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:39.714 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:39.714+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:39.815 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:39.815+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:39.901 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:39.901 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:39.915 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:39.915+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:39.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:40.017 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:40.014+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:40.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:23:40.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~6s 2024-04-18T22:23:40.106 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:40.106 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:40.117 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:40.114+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:40.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:40.194 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 44 16 96062 96046 136.414 121.312 0.00917653 0.00732574 2024-04-18T22:23:40.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:40.215+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:40.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:40.279 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:40.279 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:40.297 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:40.346 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:40.346+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:40.395 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:40.415 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:23:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:40.420 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:40.421 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:40.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:40.419+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:40.451 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:40.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:40.451 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:40.452 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:40.452 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:40.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:40.562 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:40.562+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:40.621 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:40.618+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.546626684000003, 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:40.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:40.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.546494191999997, 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:40.670 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:40.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:40.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:40.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.546422605999993, 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:40.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:40.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.546294352999993, 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:40.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:40.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.689 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.546185026000003, 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.545928688999993, 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:40.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:40.700 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.545841202999995, 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 68.545763217000001, 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:40.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:40.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.863476134999999, 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:40.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:40.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:40.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.861819756000003, 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:40.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.721 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.861695038999997, 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:40.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.859915381, 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:40.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:40.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.733 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.85943271, 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:40.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:40.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:40.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:40.739 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.859193634999997, 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:40.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:40.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.859072863999998, 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:40.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:40.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.751 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.856183452000003, 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:40.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:40.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.856061633000003, 2024-04-18T22:23:40.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:40.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.855951183999998, 2024-04-18T22:23:40.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:40.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.855881715999999, 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:40.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 34.855824575, 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:40.775 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:40.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:40.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:40.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:40.781 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:40.781 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:40.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:40.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:40.779+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:40.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:40.882+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:40.928 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:97428 5.f 5:f8aa3188:::benchmark_data_smithi032_22800_object97427:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.920719+0000", 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0083646650000000003, 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083866219999999998, 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.929 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 97428 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.920719+0000", 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.920719+0000", 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.920720+0000", 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.604e-06 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.920795+0000", 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4413999999999996e-05 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.920804+0000", 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:23:40.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.920812+0000", 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4230000000000005e-06 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.921738+0000", 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00092557000000000002 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.921880+0000", 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000142087 2024-04-18T22:23:40.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.929071+0000", 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0071915299999999998 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:97426 5.1 e41/23 5:8b4bb602:::benchmark_data_smithi032_22800_object97425:head v 41'6046, mlcod=41'6043)", 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.922503+0000", 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0065799079999999998, 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066513370000000002, 2024-04-18T22:23:40.933 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922503+0000", 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922503+0000", 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922504+0000", 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2799999999999995e-07 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922555+0000", 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0951000000000001e-05 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922568+0000", 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2629000000000001e-05 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922579+0000", 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0875e-05 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922628+0000", 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9412000000000001e-05 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.922647+0000", 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9205999999999999e-05 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:97433 5.f 5:f4c9711c:::benchmark_data_smithi032_22800_object97432:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.923881+0000", 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0052021699999999999, 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005308152, 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 97433 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.938 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.923881+0000", 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.923881+0000", 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.923882+0000", 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7699999999999992e-07 2024-04-18T22:23:40.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924027+0000", 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000145246 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924066+0000", 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8254999999999999e-05 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924072+0000", 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2570000000000004e-06 2024-04-18T22:23:40.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924409+0000", 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033712199999999998 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924534+0000", 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012549600000000001 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.941 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:97434 5.d 5:b0251eed:::benchmark_data_smithi032_22800_object97433:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.924086+0000", 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.004997479, 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0051401340000000002, 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 97434 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924086+0000", 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924086+0000", 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924086+0000", 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924122+0000", 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.587e-05 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924131+0000", 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9109999999999999e-06 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924137+0000", 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0519999999999998e-06 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924182+0000", 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5043999999999998e-05 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924341+0000", 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015831600000000001 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:97429 5.1 e41/23 5:81ef45c7:::benchmark_data_smithi032_22800_object97428:head v 41'6047, mlcod=41'6043)", 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.924198+0000", 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0048850059999999999, 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050635159999999997, 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.946 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924198+0000", 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924198+0000", 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924199+0000", 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4600000000000005e-07 2024-04-18T22:23:40.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924280+0000", 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1370000000000002e-05 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924289+0000", 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.02e-06 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924296+0000", 2024-04-18T22:23:40.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0319999999999998e-06 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924346+0000", 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0006999999999997e-05 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924358+0000", 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.151e-05 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:97435 5.2 e41/23 5:4edb5095:::benchmark_data_smithi032_22800_object97434:head v 41'6085, mlcod=41'6082)", 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.924317+0000", 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0047660139999999998, 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049749449999999997, 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924317+0000", 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924317+0000", 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924318+0000", 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7500000000000001e-07 2024-04-18T22:23:40.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924363+0000", 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5355999999999997e-05 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924370+0000", 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2389999999999996e-06 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924376+0000", 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5620000000000003e-06 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924423+0000", 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7652999999999999e-05 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924442+0000", 2024-04-18T22:23:40.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8437e-05 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:97438 5.9 5:92bd533a:::benchmark_data_smithi032_22800_object97437:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.924424+0000", 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0046591150000000001, 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049019229999999999, 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.954 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 97438 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924424+0000", 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924424+0000", 2024-04-18T22:23:40.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.924425+0000", 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2800000000000005e-07 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.925961+0000", 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015360689999999999 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.925974+0000", 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2567e-05 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.925981+0000", 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5120000000000002e-06 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926139+0000", 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000158181 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926281+0000", 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014107099999999999 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:97441 5.6 e41/19 5:6a7885a3:::benchmark_data_smithi032_22800_object97440:head v 41'6058, mlcod=41'6056)", 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.926018+0000", 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0030656580000000002, 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033443710000000001, 2024-04-18T22:23:40.958 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926018+0000", 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926018+0000", 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926021+0000", 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6200000000000001e-06 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926049+0000", 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.811e-05 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926058+0000", 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1069999999999994e-06 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926064+0000", 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.356e-06 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926118+0000", 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4360000000000001e-05 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926138+0000", 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9607e-05 2024-04-18T22:23:40.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:97432 5.1 e41/23 5:89059d7c:::benchmark_data_smithi032_22800_object97431:head v 41'6048, mlcod=41'6043)", 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.926262+0000", 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0028213140000000001, 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031312829999999999, 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926262+0000", 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926262+0000", 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926263+0000", 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2699999999999999e-07 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926397+0000", 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013412299999999999 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926407+0000", 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0533000000000001e-05 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926414+0000", 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4629999999999998e-06 2024-04-18T22:23:40.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926467+0000", 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3711000000000001e-05 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.926485+0000", 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7808999999999998e-05 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.965 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:97436 5.1 e41/23 5:841e17cf:::benchmark_data_smithi032_22800_object97435:head v 41'6049, mlcod=41'6043)", 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.927954+0000", 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00112925, 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694280000000001, 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.927954+0000", 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.927954+0000", 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.927955+0000", 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4799999999999998e-07 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928021+0000", 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6703000000000002e-05 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928033+0000", 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1378e-05 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928041+0000", 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2400000000000007e-06 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.929055+0000", 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001014369 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.929074+0000", 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8513e-05 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:97443 5.9 5:9c8faa4c:::benchmark_data_smithi032_22800_object97442:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:40.928362+0000", 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00072093999999999999, 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001100204, 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:40.969 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 97443 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928362+0000", 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928362+0000", 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:40.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928364+0000", 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.155e-06 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928446+0000", 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2900999999999998e-05 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928456+0000", 2024-04-18T22:23:40.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2879999999999998e-06 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.928461+0000", 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.831e-06 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.929084+0000", 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00062355699999999995 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:40.929230+0000", 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000146102 2024-04-18T22:23:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.973 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:40.973 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.973 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:40.973 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:40.973 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 11 2024-04-18T22:23:40.973 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:40.974 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:40.983 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:40.981+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:41.084 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:41.082+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:41.158 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:41.159 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:41.159 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:41.159 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:41.159 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:41.178 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:41.194 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 45 16 98009 97993 136.087 121.688 0.00810462 0.00734367 2024-04-18T22:23:41.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:41.218+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:41.286 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:41.285+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.249073602999999, 2024-04-18T22:23:41.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:41.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:41.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:41.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.249005005000001, 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:41.371 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:41.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248909158999993, 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:41.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:41.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248852276999997, 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.382 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:41.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:41.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.387 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248792605000006, 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:41.392 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248734643000006, 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:41.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:41.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.398 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248514951000004, 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:41.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.403 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248458021999994, 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:41.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:41.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248362135999997, 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:41.409 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:41.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:41.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.414 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248299807999999, 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.415 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:41.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:41.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.420 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248227330000006, 2024-04-18T22:23:41.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:41.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:41.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.248074861999996, 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.427 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.431 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.247996138999994, 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.432 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:41.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:41.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.246475282999995, 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:41.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:41.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:41.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.443 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 69.246376931, 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:41.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:41.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 35.564654978, 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.448 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:41.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 35.564000765000003, 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.452 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.453 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.457 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:41.458 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.459 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 35.563894070000003, 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.460 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.461 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:41.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:41.463 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 35.562028161000001, 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:41.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 35.561362838000001, 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:41.473 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:41.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:41.475 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:41.473+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:41.575 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:41.574+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:41.630 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:41.630 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:41.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.630 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98856 5.3 5:caa0096b:::benchmark_data_smithi032_22800_object98855:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.630 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.625921+0000", 2024-04-18T22:23:41.630 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0050222540000000003, 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050405190000000002, 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98856 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.625921+0000", 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.625921+0000", 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.625922+0000", 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999996e-07 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.625981+0000", 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9585e-05 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.625990+0000", 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0929999999999992e-06 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.625997+0000", 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8279999999999997e-06 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628021+0000", 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002023965 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628197+0000", 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017555899999999999 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628313+0000", 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116791 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98857 5.1 5:8f311233:::benchmark_data_smithi032_22800_object98856:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.626104+0000", 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0048391399999999996, 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0049053320000000001, 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98857 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.635 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626104+0000", 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626104+0000", 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626104+0000", 2024-04-18T22:23:41.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9400000000000005e-07 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626166+0000", 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1446999999999996e-05 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626177+0000", 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.076e-05 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626183+0000", 2024-04-18T22:23:41.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1279999999999996e-06 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626226+0000", 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3143999999999999e-05 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626302+0000", 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5908000000000006e-05 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:41.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626392+0000", 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0020999999999998e-05 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630680+0000", 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042876240000000003 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98858 5.6 5:6443e652:::benchmark_data_smithi032_22800_object98857:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.626218+0000", 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0047248619999999998, 2024-04-18T22:23:41.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048762620000000001, 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98858 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626218+0000", 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626218+0000", 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626219+0000", 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6899999999999997e-07 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626294+0000", 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5271999999999994e-05 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626302+0000", 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5680000000000002e-06 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626307+0000", 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2140000000000004e-06 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626351+0000", 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4598000000000002e-05 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626465+0000", 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114041 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626585+0000", 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000119634 2024-04-18T22:23:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630716+0000", 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041306989999999998 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98860 5.1 5:856ef7a6:::benchmark_data_smithi032_22800_object98859:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.626402+0000", 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00454098, 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047646340000000002, 2024-04-18T22:23:41.644 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98860 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626402+0000", 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626402+0000", 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626402+0000", 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2600000000000002e-07 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626452+0000", 2024-04-18T22:23:41.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9317000000000002e-05 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626459+0000", 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9970000000000001e-06 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626465+0000", 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1319999999999999e-06 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628519+0000", 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020539220000000001 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628594+0000", 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349e-05 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628693+0000", 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9272999999999996e-05 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98862 5.6 5:6c52fa39:::benchmark_data_smithi032_22800_object98861:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.626718+0000", 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0042253450000000001, 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045271369999999997, 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98862 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.649 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626718+0000", 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626718+0000", 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626718+0000", 2024-04-18T22:23:41.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9999999999999996e-07 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626802+0000", 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3405000000000005e-05 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626811+0000", 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9989999999999997e-06 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.626817+0000", 2024-04-18T22:23:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628617+0000", 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001799859 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628731+0000", 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011396 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:41.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628822+0000", 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.145e-05 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98864 5.1 5:8a421fff:::benchmark_data_smithi032_22800_object98863:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.628085+0000", 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002858167, 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032086049999999998, 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98864 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628085+0000", 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628085+0000", 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.628086+0000", 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629907+0000", 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018214360000000001 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629916+0000", 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7609999999999996e-06 2024-04-18T22:23:41.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629922+0000", 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1890000000000004e-06 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630702+0000", 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00078007300000000005 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630816+0000", 2024-04-18T22:23:41.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011369100000000001 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630947+0000", 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130925 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98865 5.4 5:271dc842:::benchmark_data_smithi032_22800_object98864:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.629934+0000", 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010087919999999999, 2024-04-18T22:23:41.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014031029999999999, 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98865 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629934+0000", 2024-04-18T22:23:41.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629934+0000", 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629935+0000", 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.75e-07 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629957+0000", 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2824e-05 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629962+0000", 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5879999999999999e-06 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629965+0000", 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.394e-06 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630030+0000", 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4493999999999996e-05 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630128+0000", 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7994999999999999e-05 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98868 5.8 5:15b78675:::benchmark_data_smithi032_22800_object98867:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.661 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.629975+0000", 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00096781700000000005, 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013969710000000001, 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98868 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629975+0000", 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629975+0000", 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629975+0000", 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0499999999999999e-07 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.629998+0000", 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2167e-05 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630005+0000", 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7330000000000003e-06 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630011+0000", 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.288e-06 2024-04-18T22:23:41.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630047+0000", 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6016000000000002e-05 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630137+0000", 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0474000000000001e-05 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98869 5.c 5.cc0a666c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.630351+0000", 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00059185599999999998, 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010529650000000001, 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98869 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630351+0000", 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630351+0000", 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630352+0000", 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.06e-07 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630420+0000", 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8096999999999995e-05 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630427+0000", 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3139999999999998e-06 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.630431+0000", 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0679999999999998e-06 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.668 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98870 5.8 5.d8ed6c98 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.631057+0000", 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998856, 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037450100000000001, 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98870 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631057+0000", 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631057+0000", 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631058+0000", 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8100000000000001e-07 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631104+0000", 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5865999999999998e-05 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631116+0000", 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1946e-05 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631125+0000", 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1130000000000007e-06 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:98871 5.e 5.d7bcda8e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:41.631181+0000", 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997625, 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027888300000000001, 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:41.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 98871 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631181+0000", 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:41.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631181+0000", 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631181+0000", 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631233+0000", 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1906000000000001e-05 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631243+0000", 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5650000000000007e-06 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:41.631249+0000", 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3690000000000003e-06 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 11 2024-04-18T22:23:41.675 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:41.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:41.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:41.674+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:41.779 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:41.777+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:41.830 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:41.831 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:41.831 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:41.831 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:41.831 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:41.851 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:41.894 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:41.893+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:41.980 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:41.977+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:42.046 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:42.046 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:42.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:42.047 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:42.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.047 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.047 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.929189464999993, 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:42.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:42.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.929056973000002, 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:42.057 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:42.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:42.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:42.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.065 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.928985386999997, 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:42.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:42.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:42.073 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.928857133999998, 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:42.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:42.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:42.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:42.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:42.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.082 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.928747806999993, 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:42.085 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.928491469999997, 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:42.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:42.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.105 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.928403983999999, 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:42.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:42.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:42.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 69.928325998000005, 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:42.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:42.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:42.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:42.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.246038916000003, 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:42.125 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.244382537, 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:42.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:42.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.132 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.244257820000001, 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.133 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.134 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:42.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:42.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:42.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.139 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.242478161999998, 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.140 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:42.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:42.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.241995490999997, 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:42.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:42.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:42.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.241756416000001, 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:42.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:42.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.160 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.241635645000002, 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:42.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:42.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.238746233000001, 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:42.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.238624414, 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:42.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:42.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:42.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.180 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.238513965000003, 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.181 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:42.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:42.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.238444497000003, 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.188 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:42.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:42.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.238387355999997, 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:42.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:42.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:42.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:42.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:42.202 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:42.202 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:42.203 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 46 16 99993 99977 135.824 124 0.0047382 0.00735775 2024-04-18T22:23:42.204 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:42.204 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:42.202+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:42.304 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:42.302+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:42.347 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:42.347 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:100297 5.6 e41/19 5:62ce4d61:::benchmark_data_smithi032_22800_object100296:head v 41'6235, mlcod=41'6231)", 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:42.348071+0000", 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00048410700000000001, 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00050004100000000001, 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348071+0000", 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:42.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348071+0000", 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348071+0000", 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9200000000000001e-07 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348136+0000", 2024-04-18T22:23:42.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4194000000000002e-05 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348144+0000", 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3820000000000004e-06 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348149+0000", 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2129999999999999e-06 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348193+0000", 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3826999999999999e-05 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:42.348205+0000", 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1562999999999999e-05 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 1 2024-04-18T22:23:42.351 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:42.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:42.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:42.410+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:42.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:42.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:23:42.506 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:42.505+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:42.554 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:42.554 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:42.554 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:42.554 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:42.554 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:42.573 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:42.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:42.611 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:42.612 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:42.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:42.618+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:42.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:42.706+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:42.726 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:42.726 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:42.747 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:42.808 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:42.805+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:42.894 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:42.894 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:42.909 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:42.905+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:42.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:43.010 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:43.006+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:43.108 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:43.108 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:43.112 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.109+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:43.126 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:43.194 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 47 16 101969 101953 135.561 123.5 0.00506617 0.00737194 2024-04-18T22:23:43.214 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.213+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:43.314 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.313+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:43.323 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:43.323 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:43.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:43.415 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.413+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:43.496 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:43.496 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:43.515 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:43.516 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:43.514+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:43.525 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [1] live_osds: [0, 2] 2024-04-18T22:23:43.525 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:43.525 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-04-18T22:23:43.526 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-18T22:23:43.526 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-18T22:23:43.526 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-04-18T22:23:43.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:43.614+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:23:43.671 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:43.671 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:43.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:43.718 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:43.718 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "" 2024-04-18T22:23:43.718 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:43.719 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.717+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:43.819 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.817+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:43.872 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:43.873 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:43.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:43.942 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:43.941+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:44.021 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:44.017+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:44.046 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:44.046 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:44.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:44.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:44.118+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:44.194 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 48 16 103968 103952 135.34 124.938 0.00523347 0.0073838 2024-04-18T22:23:44.224 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:44.221+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:44.264 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:44.265 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:44.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:44.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:44.326+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:44.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:44.426+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:44.480 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:44.480 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:44.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:44.551 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:44.550+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:44.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:44.631 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:44.629+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:44.652 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:44.652 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:44.674 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:44.734 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:44.733+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:44.798 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:44.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:44.801 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:44.835 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:44.833+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:44.866 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:44.867 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:44.867 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:44.867 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:44.867 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:44.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:44.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:44.938+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:45.037 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:45.033+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:45.044 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:45.044 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.927455199999997, 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:45.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:45.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:45.050 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.927386601999999, 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:45.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:45.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.927290756000005, 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:45.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:45.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:45.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:45.061 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.927233873999995, 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.927174202000003, 2024-04-18T22:23:45.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:45.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:45.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.071 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.927116240000004, 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:45.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:45.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926896548000002, 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:45.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.082 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926839619000006, 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:45.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926743732999995, 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:45.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:45.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926681404999997, 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:45.093 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:45.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:45.098 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926608927000004, 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926456458999994, 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:45.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.108 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.926377736000006, 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:45.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:45.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:45.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:45.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.924856879999993, 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:45.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:45.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 72.924758527999998, 2024-04-18T22:23:45.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:45.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.243036574999998, 2024-04-18T22:23:45.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:45.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.242382362000001, 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:45.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:45.132 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.242275667000001, 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.133 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.136 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.240409757999998, 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:45.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:45.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:45.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.142 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 39.239744434999999, 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.146 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.147 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:45.147 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:45.147 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:45.148 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:45.146+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:45.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:45.194 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 49 15 106091 106076 135.286 132.75 0.00600553 0.00738697 2024-04-18T22:23:45.248 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:45.245+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:45.304 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:45.304 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:45.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.304 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106295 5.c 5:39595bb1:::benchmark_data_smithi032_22800_object106294:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.304 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.300984+0000", 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0038776240000000001, 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039038300000000001, 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106295 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.300984+0000", 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.300984+0000", 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.300985+0000", 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301100+0000", 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011510699999999999 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301110+0000", 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8859999999999999e-06 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301118+0000", 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6140000000000003e-06 2024-04-18T22:23:45.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301161+0000", 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3343999999999997e-05 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301237+0000", 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6188999999999998e-05 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301319+0000", 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1798000000000005e-05 2024-04-18T22:23:45.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304118+0000", 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027989600000000001 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106296 5.2 5:402f4065:::benchmark_data_smithi032_22800_object106295:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.301176+0000", 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0036857019999999999, 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037938360000000001, 2024-04-18T22:23:45.309 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106296 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301176+0000", 2024-04-18T22:23:45.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301176+0000", 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301177+0000", 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6400000000000002e-07 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301230+0000", 2024-04-18T22:23:45.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3647000000000003e-05 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301239+0000", 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.833e-06 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301246+0000", 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3559999999999998e-06 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301282+0000", 2024-04-18T22:23:45.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6754000000000001e-05 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301420+0000", 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000137665 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.301575+0000", 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015459800000000001 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106299 5.b 5:db4c6bf1:::benchmark_data_smithi032_22800_object106298:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.303791+0000", 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001070971, 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012506489999999999, 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106299 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.314 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303791+0000", 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303791+0000", 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303791+0000", 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8299999999999997e-07 2024-04-18T22:23:45.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303897+0000", 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105751 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303908+0000", 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.131e-05 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303914+0000", 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1549999999999998e-06 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303953+0000", 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9311999999999999e-05 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304039+0000", 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5741000000000005e-05 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.317 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106301 5.6 5:6e6ff9f4:::benchmark_data_smithi032_22800_object106300:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.303984+0000", 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00087817500000000003, 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001121017, 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:45.318 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106301 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303984+0000", 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303984+0000", 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.303984+0000", 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2399999999999998e-07 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304117+0000", 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000132869 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304123+0000", 2024-04-18T22:23:45.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9699999999999996e-06 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304127+0000", 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304158+0000", 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1408999999999997e-05 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304238+0000", 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9447e-05 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304324+0000", 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5984999999999998e-05 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.322 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106302 5.8 5:159b4f42:::benchmark_data_smithi032_22800_object106301:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.304159+0000", 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00070259499999999998, 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001011828, 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106302 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304159+0000", 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304159+0000", 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304160+0000", 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2600000000000002e-07 2024-04-18T22:23:45.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304240+0000", 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0610000000000002e-05 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304248+0000", 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4939999999999997e-06 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304252+0000", 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6460000000000001e-06 2024-04-18T22:23:45.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304318+0000", 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5798999999999998e-05 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304538+0000", 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021972499999999999 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:106296 5.2 e42/23)", 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.304338+0000", 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00052361900000000002, 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00088765399999999998, 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304338+0000", 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304338+0000", 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304339+0000", 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1999999999999999e-07 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304342+0000", 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9799999999999998e-06 2024-04-18T22:23:45.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304352+0000", 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0689999999999999e-05 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304359+0000", 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1330000000000004e-06 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106303 5.b 5.5480298b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.304289+0000", 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00057273000000000003, 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00097814400000000002, 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106303 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304289+0000", 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304289+0000", 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304290+0000", 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0100000000000005e-07 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304358+0000", 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8115999999999996e-05 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304365+0000", 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5789999999999998e-06 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304372+0000", 2024-04-18T22:23:45.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3380000000000001e-06 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106304 5.7 5:efc41a7b:::benchmark_data_smithi032_22800_object106303:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.304409+0000", 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000452924, 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00091237999999999996, 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.333 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106304 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304409+0000", 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304409+0000", 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304409+0000", 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1699999999999998e-07 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304494+0000", 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4586999999999995e-05 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304503+0000", 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4880000000000007e-06 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304510+0000", 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9130000000000003e-06 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304552+0000", 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1677000000000001e-05 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304662+0000", 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109354 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304784+0000", 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012267199999999999 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106305 5.0 5:031a5589:::benchmark_data_smithi032_22800_object106304:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.304525+0000", 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00033657499999999998, 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000862508, 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106305 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304525+0000", 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304525+0000", 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304526+0000", 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2600000000000002e-07 2024-04-18T22:23:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304557+0000", 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0883000000000002e-05 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304565+0000", 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3420000000000008e-06 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304571+0000", 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2820000000000002e-06 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304715+0000", 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014323 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304832+0000", 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011742 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.341 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106306 5.2 5.4d72eae2 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.304593+0000", 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00026883000000000001, 2024-04-18T22:23:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085121600000000002, 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106306 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304593+0000", 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304593+0000", 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304594+0000", 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304634+0000", 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0669e-05 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304642+0000", 2024-04-18T22:23:45.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5449999999999998e-06 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.304649+0000", 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3010000000000001e-06 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.349 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:106284 5.2 5:401cbddf:::benchmark_data_smithi032_22800_object106283:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:45.294004+0000", 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.010857766, 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011494631, 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 106284 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.294004+0000", 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.294004+0000", 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.294005+0000", 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3799999999999996e-07 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.294057+0000", 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2188000000000002e-05 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.294066+0000", 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5900000000000008e-06 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.294070+0000", 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8470000000000003e-06 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.295684+0000", 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001613354 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.295794+0000", 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110733 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.295916+0000", 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121864 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:45.300735+0000", 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048183790000000002 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 11 2024-04-18T22:23:45.355 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:45.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:45.356 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:45.353+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:45.457 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:45.453+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:45.556 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:45.556 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:45.556 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:45.556 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:45.557 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:45.557 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:45.554+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:45.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:45.659 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:45.658+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:45.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:23:45.760 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:45.757+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:45.778 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:45.778 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:45.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:45.778 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.662331140999996, 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:45.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:45.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:45.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.662198649000004, 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:45.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:45.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:45.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:45.790 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.662127063, 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:45.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:45.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.66199881, 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:45.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:45.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:45.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.803 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.661889482999996, 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:45.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:45.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.661633146, 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:45.814 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.661545660000002, 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:45.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:45.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:45.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:45.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 73.661467673999994, 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:45.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.979180591999999, 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:45.824 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.977524213000002, 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.830 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:45.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:45.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:45.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.836 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.977399495999997, 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:45.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:45.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:45.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:45.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:45.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:45.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:45.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:45.845 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:45.846 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.975619838, 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:45.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:45.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:45.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:45.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:45.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:45.854 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.975137167, 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:45.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:45.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:45.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:45.863 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.974898091999997, 2024-04-18T22:23:45.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:45.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:45.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:45.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:45.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:45.873 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.974777320999998, 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:45.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:45.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:45.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:45.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:45.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.971887909000003, 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:45.881 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.882 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:45.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:45.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.890 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.971766090000003, 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.891 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.897 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.971655640999998, 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:45.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:45.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:45.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:45.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:45.905 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.905 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.905 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.971586172999999, 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.906 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:45.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:45.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.971529031999999, 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:45.912 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:45.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:45.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.917 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.918 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.918 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:45.918 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:45.918 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:45.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:45.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:45.918+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:46.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~12s 2024-04-18T22:23:46.020 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:46.018+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:46.069 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:107862 5.f 5:f54f1f95:::benchmark_data_smithi032_22800_object107861:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.066210+0000", 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0040918370000000001, 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0041115689999999998, 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.070 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 107862 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066210+0000", 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066210+0000", 2024-04-18T22:23:46.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066211+0000", 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4000000000000001e-07 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066285+0000", 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4133000000000003e-05 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066311+0000", 2024-04-18T22:23:46.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5474999999999999e-05 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066327+0000", 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6699999999999999e-05 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067167+0000", 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00083977800000000001 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067255+0000", 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8209999999999997e-05 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:107863 5.f 5:f199821e:::benchmark_data_smithi032_22800_object107862:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.066356+0000", 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0039465430000000003, 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040122659999999996, 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.074 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 107863 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066356+0000", 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066356+0000", 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066356+0000", 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3200000000000005e-07 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066409+0000", 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2394e-05 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066419+0000", 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0205e-05 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066425+0000", 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9569999999999999e-06 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.068727+0000", 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023018650000000002 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.068852+0000", 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012540500000000001 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:107864 5.d 5:b365a365:::benchmark_data_smithi032_22800_object107863:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.066437+0000", 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0038655640000000002, 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039690999999999997, 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.078 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 107864 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066437+0000", 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066437+0000", 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066437+0000", 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.13e-07 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066464+0000", 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.654e-05 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066474+0000", 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0139999999999999e-05 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066479+0000", 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0560000000000004e-06 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066540+0000", 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1329999999999997e-05 2024-04-18T22:23:46.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066654+0000", 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011338 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.082 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:107868 5.7 e42/23 5:e55f684f:::benchmark_data_smithi032_22800_object107867:head v 42'6815, mlcod=42'6813)", 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.067540+0000", 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0027620700000000002, 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029012719999999999, 2024-04-18T22:23:46.083 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067540+0000", 2024-04-18T22:23:46.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067540+0000", 2024-04-18T22:23:46.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067541+0000", 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5199999999999996e-07 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067618+0000", 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6747000000000002e-05 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067631+0000", 2024-04-18T22:23:46.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3619e-05 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067638+0000", 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4110000000000001e-06 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.068717+0000", 2024-04-18T22:23:46.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001079198 2024-04-18T22:23:46.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.068735+0000", 2024-04-18T22:23:46.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7955999999999999e-05 2024-04-18T22:23:46.090 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.090 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.090 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:107869 5.c e42/19 5:30ea7468:::benchmark_data_smithi032_22800_object107868:head v 42'6689, mlcod=42'6685)", 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.067754+0000", 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0025482209999999998, 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027191799999999999, 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.091 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067754+0000", 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067754+0000", 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067755+0000", 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9599999999999999e-07 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067827+0000", 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2533999999999994e-05 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067837+0000", 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6889999999999999e-06 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067843+0000", 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2829999999999998e-06 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067893+0000", 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9628e-05 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.067910+0000", 2024-04-18T22:23:46.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7011999999999999e-05 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:107870 5.3 e42/19 5:c3efbe7b:::benchmark_data_smithi032_22800_object107869:head v 42'6852, mlcod=42'6849)", 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.069179+0000", 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0011231769999999999, 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001324062, 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.095 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069179+0000", 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069179+0000", 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069180+0000", 2024-04-18T22:23:46.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2999999999999999e-07 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069246+0000", 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6373999999999998e-05 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069257+0000", 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0971999999999999e-05 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069265+0000", 2024-04-18T22:23:46.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.328e-06 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069328+0000", 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3235e-05 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.069350+0000", 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1812999999999999e-05 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:107853 5.f 5:fb48846a:::benchmark_data_smithi032_22800_object107852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.062216+0000", 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0080865999999999993, 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083210290000000006, 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 107853 2024-04-18T22:23:46.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062216+0000", 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062216+0000", 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062216+0000", 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.61e-07 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062249+0000", 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2483000000000001e-05 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062257+0000", 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9470000000000003e-06 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062264+0000", 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.742e-06 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062312+0000", 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8783999999999999e-05 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.062433+0000", 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012064499999999999 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:107861 5.f 5:f1635190:::benchmark_data_smithi032_22800_object107860:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T22:23:46.103 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.064888+0000", 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0054147040000000002, 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0056873430000000001, 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 107861 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.064888+0000", 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.064888+0000", 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.064888+0000", 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0299999999999998e-07 2024-04-18T22:23:46.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.064992+0000", 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103322 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.065001+0000", 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5440000000000004e-06 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.065007+0000", 2024-04-18T22:23:46.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4090000000000002e-06 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.065056+0000", 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9357999999999997e-05 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.065189+0000", 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000133213 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:107860 5.3 e42/19 5:c7b2e2e2:::benchmark_data_smithi032_22800_object107859:head v 42'6851, mlcod=42'6849)", 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:46.066054+0000", 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0042479120000000004, 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0045542860000000003, 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066054+0000", 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066054+0000", 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066055+0000", 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3500000000000002e-07 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066144+0000", 2024-04-18T22:23:46.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9207000000000002e-05 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066153+0000", 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6370000000000005e-06 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066161+0000", 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2199999999999992e-06 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066256+0000", 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5141000000000003e-05 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:46.066279+0000", 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2575999999999999e-05 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:23:46.111 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:46.113 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:46.121 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.117+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.195 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 50 16 108152 108136 135.155 128.75 0.00667891 0.00739404 2024-04-18T22:23:46.222 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.221+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.300 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:46.300 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:46.300 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:46.300 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:46.300 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:46.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:46.322 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.321+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.423 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.421+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.464 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:46.464 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:46.464 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:46.464 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:46.464 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.464 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.348023888, 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.465 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:46.466 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.467 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.468 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.469 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347955290000002, 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.470 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:46.471 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.472 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.473 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:46.474 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347859443999994, 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:46.476 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.477 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:46.478 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.479 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.480 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347802561999998, 2024-04-18T22:23:46.481 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.483 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.484 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.485 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.486 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347742890000006, 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:46.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.489 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:46.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347684928000007, 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:46.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:46.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:46.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347465236000005, 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.497 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:46.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.502 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347408306999995, 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:46.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:46.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:46.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.508 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347312420999998, 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:46.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:46.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347250093, 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:46.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:46.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347177615000007, 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.520 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:46.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:46.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:46.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.525 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.347025146999997, 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:46.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.346946423999995, 2024-04-18T22:23:46.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:46.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:46.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:46.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.345425567999996, 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.537 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:46.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:46.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 74.345327216000001, 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:46.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:46.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.663605263000001, 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:46.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:46.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.662951049999997, 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:46.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.662844354999997, 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:46.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.559 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.660978446000001, 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:46.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:46.564 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 40.660313123000002, 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:46.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:46.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.569 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:46.569 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:46.569 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:46.571 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:46.566+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:46.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:23:46.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:46.670+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:46.726 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.726 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109194 5.1 5:8c74c0c0:::benchmark_data_smithi032_22800_object109193:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.724042+0000", 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025512159999999998, 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025770599999999999, 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109194 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724042+0000", 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724042+0000", 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724042+0000", 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4799999999999999e-07 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724079+0000", 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6510000000000001e-05 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724084+0000", 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5029999999999997e-06 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724088+0000", 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5999999999999998e-06 2024-04-18T22:23:46.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724135+0000", 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7425000000000001e-05 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724252+0000", 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116334 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:46.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724382+0000", 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012992200000000001 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109195 5.1 5:8134d4b0:::benchmark_data_smithi032_22800_object109194:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.724308+0000", 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022851360000000001, 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023947769999999998, 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:23:46.731 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109195 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724308+0000", 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724308+0000", 2024-04-18T22:23:46.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724309+0000", 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6000000000000004e-07 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724387+0000", 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8573999999999997e-05 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724394+0000", 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724397+0000", 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8240000000000001e-06 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726587+0000", 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021895880000000001 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726713+0000", 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000126112 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109196 5.8 5:1ac28b6a:::benchmark_data_smithi032_22800_object109195:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.724885+0000", 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001708647, 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018909479999999999, 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109196 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724885+0000", 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724885+0000", 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724885+0000", 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8400000000000005e-07 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724985+0000", 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9759999999999994e-05 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724992+0000", 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0210000000000002e-06 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724996+0000", 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7799999999999998e-06 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.725033+0000", 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7194000000000003e-05 2024-04-18T22:23:46.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.725115+0000", 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2358e-05 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:109189 5.6 e43/19)", 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.726251+0000", 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00034200700000000002, 2024-04-18T22:23:46.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00056280200000000005, 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726251+0000", 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726251+0000", 2024-04-18T22:23:46.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726252+0000", 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0999999999999998e-07 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726255+0000", 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.512e-06 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726267+0000", 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1123e-05 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726274+0000", 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1879999999999996e-06 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109183 5.0 5:0ca2e199:::benchmark_data_smithi032_22800_object109182:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.742 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.719692+0000", 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0069011890000000003, 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.007170236, 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109183 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719692+0000", 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719692+0000", 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719693+0000", 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2599999999999998e-07 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719723+0000", 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0546999999999997e-05 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719728+0000", 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3539999999999999e-06 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719732+0000", 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5269999999999999e-06 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719775+0000", 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2908999999999999e-05 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.721887+0000", 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021125440000000001 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109184 5.6 5:6482001d:::benchmark_data_smithi032_22800_object109183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.719741+0000", 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0068522660000000001, 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071658379999999999, 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109184 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.747 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719741+0000", 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719741+0000", 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719742+0000", 2024-04-18T22:23:46.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9200000000000001e-07 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719783+0000", 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1165999999999999e-05 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719787+0000", 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6249999999999998e-06 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719793+0000", 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7749999999999998e-06 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719845+0000", 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1607999999999998e-05 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719987+0000", 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014196099999999999 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720132+0000", 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014481699999999999 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726799+0000", 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.006667437 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.751 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109185 5.c 5:3c89f8d9:::benchmark_data_smithi032_22800_object109184:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.719805+0000", 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0067883680000000004, 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071727479999999996, 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109185 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719805+0000", 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719805+0000", 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719805+0000", 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:46.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719860+0000", 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4802000000000001e-05 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719867+0000", 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1319999999999996e-06 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719873+0000", 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7379999999999999e-06 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719938+0000", 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5021e-05 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720045+0000", 2024-04-18T22:23:46.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010661400000000001 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720180+0000", 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013510700000000001 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724934+0000", 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047539230000000002 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109186 5.e 5:710f003d:::benchmark_data_smithi032_22800_object109185:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.719889+0000", 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0067044100000000001, 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071655369999999996, 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109186 2024-04-18T22:23:46.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719889+0000", 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719889+0000", 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719889+0000", 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.37e-07 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719926+0000", 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6279999999999998e-05 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719933+0000", 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2010000000000002e-06 2024-04-18T22:23:46.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719939+0000", 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.84e-06 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719982+0000", 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3305e-05 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720122+0000", 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014024099999999999 2024-04-18T22:23:46.760 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109187 5.5 5:aaef6d94:::benchmark_data_smithi032_22800_object109186:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.719949+0000", 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0066440450000000003, 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071802960000000001, 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.761 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109187 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719949+0000", 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719949+0000", 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719950+0000", 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3300000000000002e-07 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719984+0000", 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4527999999999999e-05 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719993+0000", 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2970000000000004e-06 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.719998+0000", 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8019999999999997e-06 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720104+0000", 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105478 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720268+0000", 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000164222 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720409+0000", 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014047800000000001 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726112+0000", 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057029730000000001 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.727112+0000", 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001000814 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109188 5.7 5:ee9c9fe9:::benchmark_data_smithi032_22800_object109187:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.720013+0000", 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.006580051, 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071644009999999999, 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109188 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.766 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720013+0000", 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720013+0000", 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720014+0000", 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5599999999999995e-07 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720051+0000", 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6871e-05 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720066+0000", 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5216e-05 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720074+0000", 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9740000000000002e-06 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720120+0000", 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6377e-05 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720240+0000", 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000119906 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720369+0000", 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012845200000000001 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.725312+0000", 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004943378 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.727103+0000", 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00179057 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.727144+0000", 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1619999999999998e-05 2024-04-18T22:23:46.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.727178+0000", 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3481e-05 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109189 5.6 5:6eaab951:::benchmark_data_smithi032_22800_object109188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.720091+0000", 2024-04-18T22:23:46.771 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0065025910000000003, 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071461859999999997, 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109189 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720091+0000", 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720091+0000", 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720091+0000", 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2099999999999996e-07 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720125+0000", 2024-04-18T22:23:46.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3652000000000002e-05 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720132+0000", 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0729999999999999e-06 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720138+0000", 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.592e-06 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.723648+0000", 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035103690000000002 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.723773+0000", 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012521599999999999 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.723914+0000", 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000141051 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.726860+0000", 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029461190000000001 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109190 5.a 5:566a4367:::benchmark_data_smithi032_22800_object109189:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.720146+0000", 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0064471060000000002, 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071382099999999999, 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109190 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720146+0000", 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720146+0000", 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720147+0000", 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720185+0000", 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8016999999999999e-05 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720192+0000", 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.720198+0000", 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.722393+0000", 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002194714 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.722476+0000", 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3104999999999998e-05 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:109193 5.b 5:d08b9ce3:::benchmark_data_smithi032_22800_object109192:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:46.723982+0000", 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026115180000000002, 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033399089999999998, 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:46.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 109193 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.723982+0000", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.723982+0000", 2024-04-18T22:23:46.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.723983+0000", 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3069999999999999e-06 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724014+0000", 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0700000000000001e-05 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:46.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724022+0000", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5709999999999998e-06 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724028+0000", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7860000000000002e-06 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724079+0000", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1028e-05 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:46.724205+0000", 2024-04-18T22:23:46.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000126174 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 13 2024-04-18T22:23:46.784 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:46.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:46.786 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.781+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:46.886 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.885+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.967 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:46.968 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:46.968 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:46.982 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:46.983 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:46.987 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:46.985+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:46.987 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:46.990 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:46.991 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:47.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:47.094 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.093+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:47.188 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.185+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:47.195 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 51 16 110085 110069 134.874 120.812 0.00758797 0.00740967 2024-04-18T22:23:47.204 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:47.204 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:47.223 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:47.289 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.285+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:47.376 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:47.377 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:47.390 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.389+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:47.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:47.491 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.489+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:47.542 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:47.542 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:47.566 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:47.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:47.610+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:47.693 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:47.690+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:47.709 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:47.709 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:47.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:47.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:47.790+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:47.882 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:47.882 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:47.894 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.893+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:47.901 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:47.995 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:47.993+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:48.096 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:48.096 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:48.097 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:48.094+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:48.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:48.195 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 52 16 112075 112059 134.672 124.375 0.00566865 0.00742071 2024-04-18T22:23:48.207 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:48.205+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:48.298 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:48.297+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:48.319 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:48.319 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:48.338 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:48.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:48.398+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:48.490 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:48.490 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:48.499 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:48.498+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:48.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:48.601 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:48.597+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:48.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:23:48.700 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:48.700 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:48.701 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:48.697+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:48.720 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:48.742 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [1] live_osds: [0, 2] 2024-04-18T22:23:48.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:48.742 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-04-18T22:23:48.742 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-18T22:23:48.742 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-18T22:23:48.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-04-18T22:23:48.802 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:48.801+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:48.867 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:48.867 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:48.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:48.895 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:48.895 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "" 2024-04-18T22:23:48.895 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:48.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:48.902+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:48.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:49.004 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:49.001+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:49.036 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:49.037 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:49.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:49.104 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:49.102+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:49.140 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:49.141 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:49.141 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:49.159 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:49.160 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:49.163 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:49.163 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:49.195 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 53 16 113930 113914 134.318 115.938 0.0141299 0.00744021 2024-04-18T22:23:49.205 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:49.201+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:49.253 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:49.253 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:49.253 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:49.253 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:49.254 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:49.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:49.307 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:49.306+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:49.408 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:49.405+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:49.475 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:49.475 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:49.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.358201668999996, 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:49.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:49.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.482 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.358069177000004, 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:49.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.357997591, 2024-04-18T22:23:49.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:49.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.495 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.357869338, 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:49.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:49.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:49.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.502 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.357760010999996, 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:49.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:49.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.508 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.357503674, 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.357416188000002, 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:49.513 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:49.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:49.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:49.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.517 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 77.357338201999994, 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.675051119999999, 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:49.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:49.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.673394741000003, 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:49.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.673270023999997, 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.671490366, 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:49.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.671007695, 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.670768619999997, 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.670647848999998, 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.667758437000003, 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.667636618000003, 2024-04-18T22:23:49.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.667526168999999, 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.667456700999999, 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:49.577 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.582 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 43.66739956, 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:49.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:49.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:49.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:49.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:49.588 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:49.590 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:49.590 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:49.585+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:49.690 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:49.689+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:114993 5.f 5:f15edbea:::benchmark_data_smithi032_22800_object114992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.734483+0000", 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0067815230000000002, 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0068045809999999997, 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:49.741 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 114993 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.734483+0000", 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.734483+0000", 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.734484+0000", 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1000000000000001e-06 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.734516+0000", 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1739999999999998e-05 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.734527+0000", 2024-04-18T22:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1269e-05 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.734534+0000", 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6250000000000001e-06 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735605+0000", 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010710260000000001 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735778+0000", 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017287100000000001 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:114994 5.7 e43/23 5:e7cb5127:::benchmark_data_smithi032_22800_object114993:head v 43'7261, mlcod=43'7258)", 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.735425+0000", 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0058393430000000003, 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059139040000000002, 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.745 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735425+0000", 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735425+0000", 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735426+0000", 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7000000000000005e-07 2024-04-18T22:23:49.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735485+0000", 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9407000000000002e-05 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735495+0000", 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7200000000000001e-06 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735501+0000", 2024-04-18T22:23:49.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3910000000000003e-06 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.737961+0000", 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002460135 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.737984+0000", 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2408e-05 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:114996 5.2 e43/23 5:4b04670b:::benchmark_data_smithi032_22800_object114995:head v 43'7207, mlcod=43'7204)", 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.735606+0000", 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0056586329999999997, 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0057654849999999999, 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735606+0000", 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735606+0000", 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735606+0000", 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7e-07 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735640+0000", 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3617e-05 2024-04-18T22:23:49.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735656+0000", 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6096e-05 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735662+0000", 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.105e-06 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.736409+0000", 2024-04-18T22:23:49.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000746515 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.736428+0000", 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9014000000000001e-05 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:114997 5.1 e43/23 5:8ac97fcd:::benchmark_data_smithi032_22800_object114996:head v 43'7146, mlcod=43'7143)", 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.735687+0000", 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0055770289999999998, 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005738188, 2024-04-18T22:23:49.752 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735687+0000", 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735687+0000", 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735688+0000", 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.37e-07 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735719+0000", 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1236999999999999e-05 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735728+0000", 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8910000000000001e-06 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735734+0000", 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1700000000000002e-06 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735882+0000", 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014737499999999999 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735897+0000", 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5569e-05 2024-04-18T22:23:49.755 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:114999 5.9 5:9bc9fd20:::benchmark_data_smithi032_22800_object114998:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.735794+0000", 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0054708689999999997, 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0056988200000000003, 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:49.756 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 114999 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735794+0000", 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735794+0000", 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735794+0000", 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1800000000000005e-07 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735821+0000", 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6726000000000001e-05 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735831+0000", 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9760000000000007e-06 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.735840+0000", 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7770000000000008e-06 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.737683+0000", 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001843318 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.737872+0000", 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00018854700000000001 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.759 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:115002 5.6 e43/19 5:6e984cbd:::benchmark_data_smithi032_22800_object115001:head v 43'7136, mlcod=43'7134)", 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.738967+0000", 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0022976199999999998, 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025942460000000001, 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.738967+0000", 2024-04-18T22:23:49.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.738967+0000", 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.738968+0000", 2024-04-18T22:23:49.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.561e-06 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739032+0000", 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3424999999999997e-05 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739044+0000", 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2481e-05 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739052+0000", 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4950000000000002e-06 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739099+0000", 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6715999999999998e-05 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739112+0000", 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3056e-05 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.763 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:115004 5.9 5:906a25db:::benchmark_data_smithi032_22800_object115003:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.739382+0000", 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018824410000000001, 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002260644, 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 115004 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.764 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739382+0000", 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739382+0000", 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739383+0000", 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.195e-06 2024-04-18T22:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739433+0000", 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9679999999999999e-05 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739444+0000", 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1571e-05 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739451+0000", 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.421e-06 2024-04-18T22:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.740057+0000", 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00060619700000000005 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.740194+0000", 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013697199999999999 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.767 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:115005 5.9 5.fbb448b9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.739464+0000", 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001800624, 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022143750000000002, 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 115005 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739464+0000", 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739464+0000", 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739464+0000", 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4099999999999998e-07 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739514+0000", 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9497999999999999e-05 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739523+0000", 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9630000000000004e-06 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739529+0000", 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5929999999999996e-06 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:115003 5.1 e43/23 5:8c7f3c26:::benchmark_data_smithi032_22800_object115002:head v 43'7147, mlcod=43'7144)", 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:49.739698+0000", 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0015664780000000001, 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00205544, 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739698+0000", 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739698+0000", 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739698+0000", 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.89e-07 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739772+0000", 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3399000000000007e-05 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739781+0000", 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5689999999999994e-06 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739789+0000", 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8900000000000007e-06 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739877+0000", 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7714999999999996e-05 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:49.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:49.739903+0000", 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6208999999999999e-05 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:23:49.774 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:49.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:49.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:49.822+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:49.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:49.890+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:49.920 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:23:49.925 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:49.925 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:49.925 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:49.925 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:49.925 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:49.943 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:49.993 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:49.989+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:50.094 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:50.093+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:50.142 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:50.143 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:50.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:50.143 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:50.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.143 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.143 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:50.144 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026617404000007, 2024-04-18T22:23:50.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:50.144 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.144 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.144 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.144 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:50.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:50.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:50.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:50.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:50.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:50.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:50.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.152 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.152 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:50.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:50.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026548805999994, 2024-04-18T22:23:50.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:50.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:50.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.02645296, 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:50.159 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:50.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:50.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:50.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:50.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:50.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:50.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:50.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:50.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:50.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:50.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:50.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:50.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.170 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026396078000005, 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:50.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026336405999999, 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.177 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:50.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:50.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.185 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026278443999999, 2024-04-18T22:23:50.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:50.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:50.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026058751999997, 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.026001823000001, 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.198 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:50.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.025905937000005, 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:50.204 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:50.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.210 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.025843609000006, 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:50.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.025771130999999, 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:50.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:50.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.025618663000003, 2024-04-18T22:23:50.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:50.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.025539940000002, 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:50.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:50.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.024019084000003, 2024-04-18T22:23:50.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:50.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:50.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:50.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.238 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 78.023920731999993, 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:50.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:50.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 44.342198779, 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:50.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:50.247 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 44.341544566000003, 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:50.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:50.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.251 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 44.341437870999997, 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:50.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:50.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 44.339571962000001, 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.256 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 44.338906639000001, 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:50.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:50.265 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:50.266 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 54 16 115907 115891 134.118 123.562 0.00372037 0.00745162 2024-04-18T22:23:50.267 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:50.262+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:50.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:50.368 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:50.365+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116319 5.e 5:792e86f0:::benchmark_data_smithi032_22800_object116318:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.409602+0000", 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.014115842999999999, 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014138199000000001, 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116319 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.409602+0000", 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.409602+0000", 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.409602+0000", 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9599999999999999e-07 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.409637+0000", 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4623999999999999e-05 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.409645+0000", 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4239999999999993e-06 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.409652+0000", 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:23:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416553+0000", 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069009789999999998 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416667+0000", 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114799 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116323 5.2 5:44c89bb4:::benchmark_data_smithi032_22800_object116322:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.411914+0000", 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011803136000000001, 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.011870594, 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116323 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.411914+0000", 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.411914+0000", 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.411915+0000", 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1399999999999997e-07 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.411935+0000", 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0021e-05 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.411942+0000", 2024-04-18T22:23:50.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5069999999999997e-06 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.411946+0000", 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2239999999999997e-06 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.412388+0000", 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000441848 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.412502+0000", 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114182 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.412639+0000", 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000137315 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416722+0000", 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0040825239999999997 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116327 5.1 5:85901406:::benchmark_data_smithi032_22800_object116326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.414728+0000", 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0089898329999999992, 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091015929999999998, 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116327 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414728+0000", 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414728+0000", 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414728+0000", 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9999999999999997e-07 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414801+0000", 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2287999999999996e-05 2024-04-18T22:23:50.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414809+0000", 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0960000000000001e-06 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414813+0000", 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5290000000000002e-06 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414853+0000", 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9193999999999998e-05 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414943+0000", 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0340999999999994e-05 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.415025+0000", 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2177999999999998e-05 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419387+0000", 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043615349999999997 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116328 5.0 5:0d007982:::benchmark_data_smithi032_22800_object116327:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.414977+0000", 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0087409259999999996, 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0088965989999999998, 2024-04-18T22:23:50.440 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116328 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414977+0000", 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414977+0000", 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.414977+0000", 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2399999999999998e-07 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.415083+0000", 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105721 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.415089+0000", 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5629999999999999e-06 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.415092+0000", 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8290000000000001e-06 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.415125+0000", 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2938000000000002e-05 2024-04-18T22:23:50.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.415212+0000", 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6376000000000001e-05 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116329 5.3 5:cb0f0c23:::benchmark_data_smithi032_22800_object116328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.416241+0000", 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0074764560000000002, 2024-04-18T22:23:50.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0076698560000000001, 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116329 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416241+0000", 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416241+0000", 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416242+0000", 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4e-07 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416389+0000", 2024-04-18T22:23:50.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014721100000000001 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416400+0000", 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0332e-05 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416407+0000", 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3100000000000003e-06 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416498+0000", 2024-04-18T22:23:50.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0692000000000005e-05 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416630+0000", 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013195599999999999 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416778+0000", 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014846099999999999 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419561+0000", 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027830709999999998 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116330 5.0 5:018fd9bd:::benchmark_data_smithi032_22800_object116329:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.416773+0000", 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0069450800000000002, 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0072090369999999997, 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.449 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116330 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416773+0000", 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416773+0000", 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416773+0000", 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.75e-07 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416823+0000", 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9628e-05 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416830+0000", 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7649999999999996e-06 2024-04-18T22:23:50.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416834+0000", 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2950000000000004e-06 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419429+0000", 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025952879999999998 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419514+0000", 2024-04-18T22:23:50.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4754999999999997e-05 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116331 5.8 5:1eb5e168:::benchmark_data_smithi032_22800_object116330:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.416846+0000", 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0068719879999999999, 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071882719999999999, 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.453 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116331 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416846+0000", 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416846+0000", 2024-04-18T22:23:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416846+0000", 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.77e-07 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416882+0000", 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5948000000000002e-05 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416887+0000", 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4940000000000003e-06 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416892+0000", 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9749999999999998e-06 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419425+0000", 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025329329999999998 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419554+0000", 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000128854 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116332 5.5 5:a6e74d3f:::benchmark_data_smithi032_22800_object116331:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.416904+0000", 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0068139890000000003, 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0072068690000000003, 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116332 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416904+0000", 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416904+0000", 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416904+0000", 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9499999999999998e-07 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416960+0000", 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6085999999999998e-05 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416965+0000", 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1789999999999999e-06 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416969+0000", 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5370000000000002e-06 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.416997+0000", 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.849e-05 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.417083+0000", 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5777000000000002e-05 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.417177+0000", 2024-04-18T22:23:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3760999999999999e-05 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419657+0000", 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024796520000000002 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116335 5.4 5:26cc3beb:::benchmark_data_smithi032_22800_object116334:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.419803+0000", 2024-04-18T22:23:50.462 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0039149179999999999, 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043950320000000001, 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116335 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419803+0000", 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419803+0000", 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419804+0000", 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4200000000000005e-07 2024-04-18T22:23:50.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419869+0000", 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5006999999999996e-05 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419878+0000", 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4229999999999994e-06 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419883+0000", 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6029999999999998e-06 2024-04-18T22:23:50.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419937+0000", 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4330999999999999e-05 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420054+0000", 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116725 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116336 5.3 5:c0fa9e7f:::benchmark_data_smithi032_22800_object116335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.419896+0000", 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003821563, 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004371687, 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116336 2024-04-18T22:23:50.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419896+0000", 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419896+0000", 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419897+0000", 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3600000000000004e-07 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419940+0000", 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3562000000000001e-05 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419946+0000", 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0090000000000004e-06 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419950+0000", 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5140000000000002e-06 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.419991+0000", 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1669999999999999e-05 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420118+0000", 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012673 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420213+0000", 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5089999999999999e-05 2024-04-18T22:23:50.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422080+0000", 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018671269999999999 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116337 5.a 5:529f52de:::benchmark_data_smithi032_22800_object116336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.420068+0000", 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0036493010000000002, 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004278619, 2024-04-18T22:23:50.472 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116337 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420068+0000", 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420068+0000", 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420069+0000", 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8400000000000004e-07 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420125+0000", 2024-04-18T22:23:50.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5633000000000001e-05 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420133+0000", 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5499999999999995e-06 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.420141+0000", 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421953+0000", 2024-04-18T22:23:50.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001812375 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422072+0000", 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118242 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116338 5.c 5:36695dcc:::benchmark_data_smithi032_22800_object116337:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.421941+0000", 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001776995, 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024414469999999998, 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116338 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421941+0000", 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421941+0000", 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421942+0000", 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0809999999999999e-06 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421981+0000", 2024-04-18T22:23:50.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9125000000000002e-05 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421989+0000", 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0020000000000006e-06 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.421994+0000", 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4600000000000002e-06 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422039+0000", 2024-04-18T22:23:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4727000000000001e-05 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422151+0000", 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112052 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422276+0000", 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012538299999999999 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116339 5.2 5:49f240af:::benchmark_data_smithi032_22800_object116338:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.422227+0000", 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014909750000000001, 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002193258, 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116339 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422227+0000", 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422227+0000", 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422227+0000", 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422314+0000", 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6590999999999998e-05 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422349+0000", 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5320000000000001e-05 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422360+0000", 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0862e-05 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422402+0000", 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2410000000000002e-05 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422543+0000", 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140951 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422707+0000", 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016412800000000001 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116340 5.7 5:ea400b3d:::benchmark_data_smithi032_22800_object116339:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.422370+0000", 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0013480549999999999, 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020918590000000002, 2024-04-18T22:23:50.485 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116340 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422370+0000", 2024-04-18T22:23:50.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422370+0000", 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422370+0000", 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5300000000000004e-07 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422464+0000", 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4129000000000006e-05 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422490+0000", 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5440999999999999e-05 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422496+0000", 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8089999999999998e-06 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422554+0000", 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8899e-05 2024-04-18T22:23:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422699+0000", 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014439699999999999 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422834+0000", 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013485600000000001 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116341 5.6 5:650b4924:::benchmark_data_smithi032_22800_object116340:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.422518+0000", 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0011998340000000001, 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019814110000000002, 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116341 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422518+0000", 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422518+0000", 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422518+0000", 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1999999999999999e-07 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422568+0000", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9996e-05 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422577+0000", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1110000000000008e-06 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422583+0000", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5200000000000003e-06 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422627+0000", 2024-04-18T22:23:50.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3560000000000003e-05 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422744+0000", 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116928 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422871+0000", 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012707899999999999 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:116342 5.b 5:d5f42003:::benchmark_data_smithi032_22800_object116341:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:50.422600+0000", 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0011172199999999999, 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019363220000000001, 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 116342 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422600+0000", 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422600+0000", 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422601+0000", 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7400000000000003e-07 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422629+0000", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8197000000000001e-05 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422637+0000", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0579999999999998e-06 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422644+0000", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2519999999999996e-06 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422714+0000", 2024-04-18T22:23:50.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0582999999999998e-05 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:50.422855+0000", 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140545 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 16 2024-04-18T22:23:50.497 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:50.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:50.494+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:50.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:50.600 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:50.597+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:50.691 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:50.691 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:50.691 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:50.691 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:50.692 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:50.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:50.698+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:50.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:50.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:50.798+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:50.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:50.898+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:50.908 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791940538000006, 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:50.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:50.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:50.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791808046, 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:50.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:50.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791736459999996, 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:50.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:50.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:50.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791608206999996, 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.927 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791498880000006, 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:50.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791242542999996, 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791155056999997, 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:50.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:50.948 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 78.791077071000004, 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:50.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.108789989000002, 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:50.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.107133609999998, 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.107008893, 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.105229235000003, 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.975 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.104746564000003, 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:50.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:50.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.982 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.104507489, 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:50.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:50.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.104386718000001, 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:50.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.101497305999999, 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:50.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:50.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:50.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:51.000 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.101375486999999, 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:51.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:51.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:51.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:51.006 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.101265038000001, 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:51.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:51.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:51.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.101195570000002, 2024-04-18T22:23:51.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:51.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:51.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 45.101138429000002, 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:51.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:51.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:51.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:51.023 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.024 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:51.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:51.024 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:51.024 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:51.025 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.022+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:51.126 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.122+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:51.182 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:117843 5.f 5:f8725ce0:::benchmark_data_smithi032_22800_object117842:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:51.182 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.174707+0000", 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00791678, 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079460759999999998, 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 117843 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174707+0000", 2024-04-18T22:23:51.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174707+0000", 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174708+0000", 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.035e-06 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174793+0000", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5297999999999995e-05 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174812+0000", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8641999999999999e-05 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174819+0000", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4839999999999998e-06 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.174879+0000", 2024-04-18T22:23:51.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.906e-05 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.175062+0000", 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00018316400000000001 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:117840 5.7 e44/23 5:e7922339:::benchmark_data_smithi032_22800_object117839:head v 44'7427, mlcod=44'7424)", 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.176270+0000", 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0063540589999999996, 2024-04-18T22:23:51.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0064561180000000003, 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176270+0000", 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176270+0000", 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176270+0000", 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1699999999999998e-07 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176345+0000", 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5012000000000006e-05 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176355+0000", 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8360000000000004e-06 2024-04-18T22:23:51.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176362+0000", 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9210000000000003e-06 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176413+0000", 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0859999999999998e-05 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.176431+0000", 2024-04-18T22:23:51.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7971000000000001e-05 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:117848 5.f 5:f337fda7:::benchmark_data_smithi032_22800_object117847:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.177918+0000", 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0047061150000000003, 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0048686930000000003, 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:51.190 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 117848 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.177918+0000", 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.177918+0000", 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.177919+0000", 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.161e-06 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.177981+0000", 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2178e-05 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.177995+0000", 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3688e-05 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178000+0000", 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3129999999999998e-06 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178061+0000", 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1444000000000003e-05 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178232+0000", 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000170366 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.193 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:117846 5.c e44/19 5:37680d79:::benchmark_data_smithi032_22800_object117845:head v 44'7320, mlcod=44'7316)", 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.178307+0000", 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0043168640000000001, 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0045398670000000004, 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178307+0000", 2024-04-18T22:23:51.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178307+0000", 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178307+0000", 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178368+0000", 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0473999999999997e-05 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178375+0000", 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.81e-06 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178379+0000", 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9389999999999999e-06 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178418+0000", 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9644000000000002e-05 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.178431+0000", 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2566e-05 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:117850 5.9 5:9aee27c8:::benchmark_data_smithi032_22800_object117849:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.179115+0000", 2024-04-18T22:23:51.197 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003508929, 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037921579999999999, 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 117850 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179115+0000", 2024-04-18T22:23:51.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179115+0000", 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179116+0000", 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6700000000000002e-07 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179225+0000", 2024-04-18T22:23:51.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010922 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179237+0000", 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2503999999999999e-05 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179245+0000", 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6890000000000004e-06 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179301+0000", 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6039999999999999e-05 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.179405+0000", 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103473 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:117849 5.2 e44/23 5:46db8271:::benchmark_data_smithi032_22800_object117848:head v 44'7404, mlcod=44'7402)", 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.180198+0000", 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0024260480000000001, 2024-04-18T22:23:51.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027703580000000001, 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180198+0000", 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180198+0000", 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180198+0000", 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.13e-07 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180305+0000", 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106832 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180313+0000", 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.028e-06 2024-04-18T22:23:51.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180318+0000", 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.899e-06 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180398+0000", 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0146000000000001e-05 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180418+0000", 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9551999999999998e-05 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.204 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:117852 5.f 5:f9dcaa81:::benchmark_data_smithi032_22800_object117851:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.180208+0000", 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0024160330000000002, 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0028175069999999999, 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 117852 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.205 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180208+0000", 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180208+0000", 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180208+0000", 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7899999999999999e-07 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180380+0000", 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000171831 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180389+0000", 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1370000000000008e-06 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180396+0000", 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0949999999999998e-06 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180632+0000", 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00023577 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.180940+0000", 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030795799999999999 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:117854 5.2 e44/23 5:47f80f0c:::benchmark_data_smithi032_22800_object117853:head v 44'7405, mlcod=44'7402)", 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.182115+0000", 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00050852099999999995, 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097226400000000003, 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182115+0000", 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182115+0000", 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182116+0000", 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4300000000000002e-07 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182167+0000", 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1360999999999998e-05 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182183+0000", 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5474000000000001e-05 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182193+0000", 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0036e-05 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182633+0000", 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00043972199999999998 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182645+0000", 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2571999999999999e-05 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:117858 5.5 e44/19 5:a0ec9023:::benchmark_data_smithi032_22800_object117857:head v 44'7280, mlcod=44'7278)", 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:51.182353+0000", 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00027090600000000001, 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00078757199999999995, 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:51.212 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182353+0000", 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182353+0000", 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182354+0000", 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:23:51.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182422+0000", 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8102000000000006e-05 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182442+0000", 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0000000000000002e-05 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182452+0000", 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.011e-05 2024-04-18T22:23:51.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182506+0000", 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4577000000000003e-05 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:51.182527+0000", 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0831000000000001e-05 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:51.215 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:23:51.216 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:51.216 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 55 16 117892 117876 133.935 124.062 0.00746865 0.00746169 2024-04-18T22:23:51.217 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:51.228 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:51.225+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:51.329 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:51.325+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:51.333 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:51.354 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:51.357 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:51.357 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:51.407 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:51.408 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:51.427 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:51.430 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.426+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.530 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.526+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.572 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:51.572 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:51.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:51.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.638+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.732 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.730+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:51.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:51.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~18s 2024-04-18T22:23:51.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.830+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.929 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:51.929 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:51.933 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:51.930+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:51.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:23:51.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:52.034 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:52.033+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:52.099 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:52.099 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:52.119 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:52.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:52.161+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:52.195 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 56 16 119899 119883 133.783 125.438 0.00516974 0.0074699 2024-04-18T22:23:52.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:52.234+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:52.273 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:52.274 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:52.296 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:52.336 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:52.333+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:52.437 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:52.433+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:52.490 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:52.490 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:52.508 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:52.538 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:52.537+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:52.639 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:52.637+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:52.657 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:52.657 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:52.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:52.740 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:52.738+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:52.828 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:52.828 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:52.841 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:52.838+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:52.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:23:52.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:52.938+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:53.002 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:53.002 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:53.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:53.043 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:53.038+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:53.144 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.141+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.171 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:53.171 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:53.193 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:53.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 57 16 121847 121831 133.572 121.75 0.00554698 0.00748154 2024-04-18T22:23:53.245 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:53.242+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:53.345 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:53.345 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:53.346 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.341+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:53.363 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:53.447 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.445+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.519 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:53.520 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:53.538 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:53.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:53.542 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:53.548 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.545+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.552 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:53.552 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:53.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:53.649 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.645+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.721 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.605351240999994, 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.722 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:53.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.727 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.605282642999995, 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:53.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:53.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.605186797000002, 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:53.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.737 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.738 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.605129915000006, 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:53.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.605070243, 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:53.744 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:53.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:53.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:53.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:53.749 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.605012281, 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:53.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:53.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:53.755 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604792588999999, 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:53.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604735660000003, 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:53.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:53.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604639774000006, 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.766 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:53.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:53.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:53.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604577445999993, 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:53.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604504968000001, 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:53.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:53.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604352500000005, 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.783 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.784 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:53.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:53.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.604273777000003, 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.789 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:53.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.794 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.602752921000004, 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:53.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 81.602654568999995, 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:53.800 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:53.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:53.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.920932616000002, 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.804 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:53.808 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.920278402999998, 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:53.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:53.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:53.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.920171707999998, 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.813 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:53.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.918305799000002, 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:53.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:53.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 47.917640476000003, 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:53.822 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:53.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:53.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:53.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:53.826 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:53.827 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.825+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:53.916 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [1] live_osds: [0, 2] 2024-04-18T22:23:53.917 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:53.917 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-18T22:23:53.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-18T22:23:53.928 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:53.925+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:53.988 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:53.988 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:53.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.988 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:123409 5.1 e45/23)", 2024-04-18T22:23:53.988 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.984988+0000", 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0027799740000000002, 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00279839, 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984988+0000", 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984988+0000", 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984988+0000", 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7899999999999999e-07 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984992+0000", 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4209999999999999e-06 2024-04-18T22:23:53.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985000+0000", 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.259e-06 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985007+0000", 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6159999999999996e-06 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:53.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123418 5.3 5:c5113221:::benchmark_data_smithi032_22800_object123417:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.984897+0000", 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002870428, 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029488209999999999, 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123418 2024-04-18T22:23:53.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984897+0000", 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984897+0000", 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984898+0000", 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9299999999999997e-07 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985055+0000", 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015695300000000001 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985063+0000", 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1929999999999993e-06 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985069+0000", 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999999e-06 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985120+0000", 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0859000000000002e-05 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985286+0000", 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016633199999999999 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985437+0000", 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015124600000000001 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:123412 5.2 e45/23)", 2024-04-18T22:23:53.996 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.985825+0000", 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0019429989999999999, 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020873760000000002, 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985825+0000", 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985825+0000", 2024-04-18T22:23:53.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985825+0000", 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1799999999999995e-07 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985828+0000", 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.779e-06 2024-04-18T22:23:53.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985837+0000", 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4199999999999996e-06 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.985843+0000", 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2739999999999998e-06 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:53.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986671+0000", 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00082881700000000003 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:123411 5.1 e45/23)", 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.986911+0000", 2024-04-18T22:23:54.000 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00085716899999999996, 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001046618, 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986911+0000", 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986911+0000", 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986911+0000", 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1100000000000001e-07 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986914+0000", 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8770000000000002e-06 2024-04-18T22:23:54.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986924+0000", 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0631e-05 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986929+0000", 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4649999999999996e-06 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123419 5.4 5:28aaae8d:::benchmark_data_smithi032_22800_object123418:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.987152+0000", 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00061538499999999998, 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00085358199999999997, 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.005 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123419 2024-04-18T22:23:54.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.005 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987152+0000", 2024-04-18T22:23:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987152+0000", 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987153+0000", 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999996e-07 2024-04-18T22:23:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987346+0000", 2024-04-18T22:23:54.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000192757 2024-04-18T22:23:54.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987354+0000", 2024-04-18T22:23:54.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3639999999999999e-06 2024-04-18T22:23:54.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987361+0000", 2024-04-18T22:23:54.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0310000000000001e-06 2024-04-18T22:23:54.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987490+0000", 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012854199999999999 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987615+0000", 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125456 2024-04-18T22:23:54.011 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.012 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.012 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123420 5.e 5:7889f144:::benchmark_data_smithi032_22800_object123419:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.013 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.987381+0000", 2024-04-18T22:23:54.013 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00038632800000000002, 2024-04-18T22:23:54.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00069510800000000005, 2024-04-18T22:23:54.013 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:54.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123420 2024-04-18T22:23:54.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.014 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987381+0000", 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987381+0000", 2024-04-18T22:23:54.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987382+0000", 2024-04-18T22:23:54.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699999999999997e-07 2024-04-18T22:23:54.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987410+0000", 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8447999999999999e-05 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987418+0000", 2024-04-18T22:23:54.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3649999999999998e-06 2024-04-18T22:23:54.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987424+0000", 2024-04-18T22:23:54.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9050000000000002e-06 2024-04-18T22:23:54.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987505+0000", 2024-04-18T22:23:54.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1218999999999996e-05 2024-04-18T22:23:54.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987596+0000", 2024-04-18T22:23:54.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0682999999999999e-05 2024-04-18T22:23:54.020 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.021 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.021 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.021 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123421 5.1 5.7736d301 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.021 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.987644+0000", 2024-04-18T22:23:54.022 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00012370299999999999, 2024-04-18T22:23:54.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00049005399999999997, 2024-04-18T22:23:54.022 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:54.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123421 2024-04-18T22:23:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987644+0000", 2024-04-18T22:23:54.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987644+0000", 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987645+0000", 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9999999999999996e-07 2024-04-18T22:23:54.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987726+0000", 2024-04-18T22:23:54.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1528999999999998e-05 2024-04-18T22:23:54.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987735+0000", 2024-04-18T22:23:54.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9870000000000005e-06 2024-04-18T22:23:54.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987742+0000", 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8539999999999999e-06 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.028 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123405 5.7 5:e50871aa:::benchmark_data_smithi032_22800_object123404:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.977993+0000", 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0097749199999999994, 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010194282000000001, 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123405 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.977993+0000", 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.977993+0000", 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.977993+0000", 2024-04-18T22:23:54.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6000000000000004e-07 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978025+0000", 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1260999999999998e-05 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978030+0000", 2024-04-18T22:23:54.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1070000000000004e-06 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978034+0000", 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5229999999999997e-06 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984617+0000", 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0065824799999999999 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984740+0000", 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012322799999999999 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984858+0000", 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117903 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987167+0000", 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002308821 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123422 5.6 5:65a9213c:::benchmark_data_smithi032_22800_object123421:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.987754+0000", 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 1.3575e-05, 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00050745800000000002, 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123422 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.035 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987754+0000", 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987754+0000", 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987755+0000", 2024-04-18T22:23:54.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4899999999999995e-07 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987789+0000", 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4749999999999998e-05 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987799+0000", 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6209999999999999e-06 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987805+0000", 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7509999999999997e-06 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.988111+0000", 2024-04-18T22:23:54.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00030649900000000001 2024-04-18T22:23:54.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.988266+0000", 2024-04-18T22:23:54.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000155095 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123423 5.4 5.292aa614 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.987814+0000", 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999537, 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00050760699999999998, 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:54.040 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.041 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.041 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123423 2024-04-18T22:23:54.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.041 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987814+0000", 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987814+0000", 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987814+0000", 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0900000000000002e-07 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987849+0000", 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4418999999999997e-05 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987858+0000", 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7930000000000003e-06 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987863+0000", 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1159999999999998e-06 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123424 5.2 5.545e2cf2 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.987871+0000", 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.999897, 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00049915799999999998, 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.044 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123424 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987871+0000", 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987871+0000", 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987871+0000", 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3099999999999998e-07 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987904+0000", 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2740999999999997e-05 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987912+0000", 2024-04-18T22:23:54.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1419999999999993e-06 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987917+0000", 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9699999999999998e-06 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123406 5.7 5:e71dacf2:::benchmark_data_smithi032_22800_object123405:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.978248+0000", 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0095200040000000003, 2024-04-18T22:23:54.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010174692000000001, 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123406 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978248+0000", 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978248+0000", 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978248+0000", 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9699999999999995e-07 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978307+0000", 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8375000000000003e-05 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978315+0000", 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7369999999999995e-06 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978322+0000", 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5880000000000003e-06 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987313+0000", 2024-04-18T22:23:54.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0089908079999999994 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987432+0000", 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011934699999999999 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.987568+0000", 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000135928 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123407 5.b 5:d8630ffb:::benchmark_data_smithi032_22800_object123406:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.978393+0000", 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0093748270000000005, 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.01010255, 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123407 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978393+0000", 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978393+0000", 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978393+0000", 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7899999999999998e-07 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978470+0000", 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6264e-05 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978477+0000", 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.978483+0000", 2024-04-18T22:23:54.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.575e-06 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.979751+0000", 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012678710000000001 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.979871+0000", 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012024299999999999 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123409 5.1 5:83216e05:::benchmark_data_smithi032_22800_object123408:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.980093+0000", 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0076747760000000003, 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0084647779999999992, 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123409 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980093+0000", 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980093+0000", 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980094+0000", 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2950000000000001e-06 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980127+0000", 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2957999999999999e-05 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980137+0000", 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7790000000000007e-06 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980144+0000", 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2289999999999997e-06 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980201+0000", 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6975999999999998e-05 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980318+0000", 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011668200000000001 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980440+0000", 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012207099999999999 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.988401+0000", 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0079616089999999997 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123411 5.1 5:8c399c82:::benchmark_data_smithi032_22800_object123410:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.980235+0000", 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0075328030000000002, 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.008396166, 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123411 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980235+0000", 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980235+0000", 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980235+0000", 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7299999999999996e-07 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980261+0000", 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5448000000000001e-05 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.062 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980267+0000", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2940000000000002e-06 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.980271+0000", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7960000000000002e-06 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982374+0000", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021033750000000002 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.063 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982509+0000", 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013437799999999999 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982652+0000", 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014319499999999999 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.988438+0000", 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057862249999999999 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123416 5.1 5:8c9bf536:::benchmark_data_smithi032_22800_object123415:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.982385+0000", 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0053822929999999998, 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063246029999999998, 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123416 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982385+0000", 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982385+0000", 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982386+0000", 2024-04-18T22:23:54.066 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6499999999999999e-07 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982410+0000", 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4513000000000001e-05 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982416+0000", 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5509999999999999e-06 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.067 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982420+0000", 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8589999999999998e-06 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984644+0000", 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022239400000000002 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984760+0000", 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116124 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.068 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.984917+0000", 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157264 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:123417 5.1 5:8c56641e:::benchmark_data_smithi032_22800_object123416:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:53.982693+0000", 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0050746799999999998, 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060840010000000003, 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 123417 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982693+0000", 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982693+0000", 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982694+0000", 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.057e-06 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982799+0000", 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104916 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982808+0000", 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5219999999999996e-06 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.982815+0000", 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8820000000000003e-06 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986365+0000", 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003549983 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986506+0000", 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014092299999999999 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:53.986665+0000", 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000158278 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 17 2024-04-18T22:23:54.073 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:54.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:54.075 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:54.073+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:no valid command found; 10 closest matches: 2024-04-18T22:23:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:0 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:1 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:2 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:abort 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:assert 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:bench [] [] [] [] 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:bluefs debug_inject_read_zeros 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:bluefs files list 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:bluefs stats 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:bluestore allocator dump block 2024-04-18T22:23:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:Error EINVAL: invalid command 2024-04-18T22:23:54.094 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T22:23:54.095 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T22:23:54.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-18T22:23:54.176 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:54.173+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:54.195 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 58 16 123855 123839 133.432 125.5 0.0105288 0.00748947 2024-04-18T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:54.222 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:54.222 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T22:23:54.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-18T22:23:54.276 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:54.277 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:54.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:54.274+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:54.297 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:no valid command found; 10 closest matches: 2024-04-18T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:0 2024-04-18T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:1 2024-04-18T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2 2024-04-18T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:abort 2024-04-18T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:assert 2024-04-18T22:23:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:bench [] [] [] [] 2024-04-18T22:23:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:bluefs debug_inject_read_zeros 2024-04-18T22:23:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:bluefs files list 2024-04-18T22:23:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:bluefs stats 2024-04-18T22:23:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:bluestore allocator dump block 2024-04-18T22:23:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:Error EINVAL: invalid command 2024-04-18T22:23:54.379 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T22:23:54.379 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T22:23:54.382 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:54.381+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:54.480 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:54.477+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:54.495 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:54.496 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.376016766000006, 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:54.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:54.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:54.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:54.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:54.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375884274000001, 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:54.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375812687999996, 2024-04-18T22:23:54.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:54.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:54.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:54.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:54.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:54.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:54.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:54.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:54.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:54.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.529 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.530 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.530 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.530 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375684434999997, 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:54.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:54.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:54.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:54.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:54.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:54.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:54.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:54.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.542 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375575108000007, 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:54.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:54.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375318770999996, 2024-04-18T22:23:54.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:54.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:54.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:54.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.558 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375231284999998, 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:54.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:54.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 82.375153299000004, 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:54.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:54.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.568 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.692866217000002, 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.691209837999999, 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.574 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.691085121, 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:54.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.689305462999997, 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:54.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.688822792000003, 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:54.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.688583717, 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:54.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:54.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:54.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.688462946000001, 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.685573534, 2024-04-18T22:23:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.685451714999999, 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.615 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:54.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:54.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.685341266000002, 2024-04-18T22:23:54.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:54.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.685271798000002, 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:54.627 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.685214657000003, 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.633 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:54.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:54.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:54.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:54.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:54.638 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:54.640 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:54.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:54.638+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:54.740 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:54.737+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:125119 5.c e45/19 5:3b1d565d:::benchmark_data_smithi032_22800_object125118:head v 45'7827, mlcod=45'7825)", 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:54.799205+0000", 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0016519900000000001, 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001668728, 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:54.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799205+0000", 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799205+0000", 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799206+0000", 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8899999999999999e-07 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799277+0000", 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1679000000000003e-05 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799286+0000", 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1540000000000008e-06 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799291+0000", 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6430000000000003e-06 2024-04-18T22:23:54.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799340+0000", 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8711999999999997e-05 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:54.799362+0000", 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2160000000000001e-05 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 1 2024-04-18T22:23:54.803 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:54.817 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:54.859 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:54.858+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:54.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:54.938+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:54.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:23:55.010 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:55.010 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:55.010 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:55.010 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:55.010 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:55.031 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:55.042 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:55.041+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:55.143 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:55.141+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:55.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 59 16 126009 125993 133.452 134.625 0.00879022 0.00748859 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.109502414000005, 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.226 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:55.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:55.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.109433816000006, 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:55.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:55.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:55.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:55.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.109337969999999, 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:55.237 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.242 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.109281088000003, 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:55.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.109221415999997, 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:55.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:55.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:55.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:55.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.109163453999997, 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:55.253 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:55.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.258 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.108943761999996, 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:55.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.108886833, 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:55.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:55.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:55.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.108790947000003, 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:55.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.108728619000004, 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.276 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.281 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.108656140999997, 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:55.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:55.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.108503673000001, 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:55.287 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:55.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:55.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.292 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.10842495, 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:55.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.106904094000001, 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:55.302 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.106805742000006, 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:55.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 49.425083788999999, 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.307 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:55.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:55.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 49.424429576000001, 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:55.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.315 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 49.424322881000002, 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:55.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 49.422456971999999, 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.320 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 49.421791648999999, 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:55.324 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:55.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:55.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:55.329 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:55.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:55.326+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:55.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:55.431 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:55.430+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126614 5.c 5:3f0d8a40:::benchmark_data_smithi032_22800_object126613:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.479870+0000", 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0083546939999999993, 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0083738179999999999, 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126614 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479870+0000", 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479870+0000", 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479871+0000", 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7699999999999992e-07 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479900+0000", 2024-04-18T22:23:55.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8983999999999999e-05 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479911+0000", 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0883000000000001e-05 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479919+0000", 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3489999999999992e-06 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479970+0000", 2024-04-18T22:23:55.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1184999999999999e-05 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.480123+0000", 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015294199999999999 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.480248+0000", 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012460999999999999 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483479+0000", 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032308060000000001 2024-04-18T22:23:55.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126619 5.1 5:8045ee66:::benchmark_data_smithi032_22800_object126618:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.483692+0000", 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0045324609999999998, 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046023990000000001, 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126619 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483692+0000", 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483692+0000", 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483693+0000", 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1699999999999997e-07 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483758+0000", 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4578000000000004e-05 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483771+0000", 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3583e-05 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483779+0000", 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3080000000000008e-06 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483837+0000", 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7933999999999998e-05 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483968+0000", 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013020499999999999 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484071+0000", 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103529 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486774+0000", 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002702931 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126620 5.c 5:30e3fa57:::benchmark_data_smithi032_22800_object126619:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.483797+0000", 2024-04-18T22:23:55.497 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00442764, 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045486190000000003, 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126620 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483797+0000", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483797+0000", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483798+0000", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0399999999999996e-07 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483859+0000", 2024-04-18T22:23:55.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1598000000000002e-05 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483868+0000", 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3500000000000003e-06 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483877+0000", 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3329999999999997e-06 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484122+0000", 2024-04-18T22:23:55.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00024471200000000001 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486098+0000", 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019759679999999998 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486204+0000", 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106603 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126623 5.2 5:4e52a221:::benchmark_data_smithi032_22800_object126622:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.484087+0000", 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0041380610000000002, 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043412030000000001, 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126623 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484087+0000", 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484087+0000", 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484087+0000", 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6199999999999996e-07 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484125+0000", 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7796000000000002e-05 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484136+0000", 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0929e-05 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484143+0000", 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0899999999999999e-06 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484200+0000", 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6601000000000002e-05 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484353+0000", 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015328 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484499+0000", 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014559299999999999 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126625 5.4 5:24318269:::benchmark_data_smithi032_22800_object126624:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.484589+0000", 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003635503, 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039221760000000003, 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126625 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484589+0000", 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484589+0000", 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484590+0000", 2024-04-18T22:23:55.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0100000000000004e-07 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484653+0000", 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3199000000000003e-05 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484662+0000", 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3870000000000001e-06 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484671+0000", 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2409999999999995e-06 2024-04-18T22:23:55.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.488340+0000", 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036696110000000001 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.488462+0000", 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012159699999999999 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126626 5.4 5.e771f3b4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.484860+0000", 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0033650419999999999, 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037224430000000002, 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126626 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484860+0000", 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484860+0000", 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484860+0000", 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5700000000000002e-07 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484962+0000", 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000102099 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484972+0000", 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7200000000000001e-06 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.484979+0000", 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6769999999999998e-06 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126627 5.c 5.1ab4bb1c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.485194+0000", 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030301799999999999, 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003445568, 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126627 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485194+0000", 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485194+0000", 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485195+0000", 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9999999999999996e-07 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485287+0000", 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1658999999999998e-05 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485296+0000", 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7669999999999992e-06 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485302+0000", 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1630000000000001e-06 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126628 5.6 5:604d4682:::benchmark_data_smithi032_22800_object126627:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.485583+0000", 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026418959999999999, 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031202859999999999, 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126628 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485583+0000", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485583+0000", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485583+0000", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9400000000000005e-07 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485678+0000", 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4450999999999994e-05 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485688+0000", 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0281e-05 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485694+0000", 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0480000000000004e-06 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485748+0000", 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3369999999999999e-05 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485848+0000", 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001002 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.485940+0000", 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2194999999999994e-05 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:55.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.488468+0000", 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002528457 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126629 5.3 5:c72b582a:::benchmark_data_smithi032_22800_object126628:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.486800+0000", 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001424486, 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019445479999999999, 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126629 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486800+0000", 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486800+0000", 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486801+0000", 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6799999999999999e-07 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486864+0000", 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2961999999999998e-05 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486874+0000", 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5899999999999997e-06 2024-04-18T22:23:55.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486880+0000", 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7259999999999996e-06 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.486925+0000", 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5080999999999997e-05 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487044+0000", 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118385 2024-04-18T22:23:55.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487122+0000", 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8070000000000003e-05 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126630 5.2 5:4586eac6:::benchmark_data_smithi032_22800_object126629:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.487197+0000", 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001028052, 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00160293, 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126630 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487197+0000", 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487197+0000", 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487197+0000", 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0100000000000004e-07 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487283+0000", 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5555000000000003e-05 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487300+0000", 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6840000000000001e-05 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487307+0000", 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0770000000000002e-06 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487354+0000", 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7561000000000001e-05 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487484+0000", 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013008099999999999 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.487608+0000", 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012324800000000001 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126605 5.5 5:a168dcf3:::benchmark_data_smithi032_22800_object126604:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.476718+0000", 2024-04-18T22:23:55.528 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011506517000000001, 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012139735, 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126605 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476718+0000", 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476718+0000", 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476719+0000", 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476754+0000", 2024-04-18T22:23:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5413000000000001e-05 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476762+0000", 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.644e-06 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476767+0000", 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6879999999999996e-06 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.478943+0000", 2024-04-18T22:23:55.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021756509999999998 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479073+0000", 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012954899999999999 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.479213+0000", 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000140323 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483465+0000", 2024-04-18T22:23:55.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042524889999999999 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:126606 5.5 5:a0970b54:::benchmark_data_smithi032_22800_object126605:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:55.476778+0000", 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011446379, 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012130521, 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:55.533 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 126606 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476778+0000", 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476778+0000", 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476779+0000", 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6300000000000005e-07 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476812+0000", 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2999e-05 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476820+0000", 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4209999999999995e-06 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.476826+0000", 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0179999999999998e-06 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.481192+0000", 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043659199999999997 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.481329+0000", 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013712899999999999 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.481430+0000", 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100515 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:55.483498+0000", 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020678480000000002 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 12 2024-04-18T22:23:55.537 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:55.539 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:55.539 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:55.537+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:55.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:55.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:55.638+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:55.711 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:55.711 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:55.711 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:55.730 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:55.732 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:55.733 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:55.741 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:55.741 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:55.741 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:55.738+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:55.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:55.842 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:55.838+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:55.921 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:55.921 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:23:55.942 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:55.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:55.938+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.043 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.042+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.095 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.095 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.118 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:56.159 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.158+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:2024-04-18T22:23:56.197744+0000 min lat: 0.00133878 max lat: 1.06735 avg lat: 0.00748971 2024-04-18T22:23:56.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-18T22:23:56.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout: 60 3 128111 128108 133.431 132.188 0.00534231 0.00748971 2024-04-18T22:23:56.245 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.242+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.279 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.279 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.292 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:56.347 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:56.345+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:56.448 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:56.445+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:56.457 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.457 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:56.548 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.546+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.591 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.592 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:56.649 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.646+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.720 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.720 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:56.750 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.746+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.838 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.838 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:56.851 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:56.849+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:56.951 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:56.950+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:56.981 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:56.981 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:56.995 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:57.083 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:57.082+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:57.153 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:57.150+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:57.157 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:57.157 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:57.171 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:57.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Total time run: 60.0071 2024-04-18T22:23:57.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Total writes made: 128111 2024-04-18T22:23:57.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Write size: 65536 2024-04-18T22:23:57.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Object size: 65536 2024-04-18T22:23:57.196 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Bandwidth (MB/sec): 133.433 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Stddev Bandwidth: 31.3234 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Max bandwidth (MB/sec): 238.25 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Min bandwidth (MB/sec): 20.75 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Average IOPS: 2134 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Stddev IOPS: 501.175 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Max IOPS: 3812 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Min IOPS: 332 2024-04-18T22:23:57.197 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Average Latency(s): 0.00748972 2024-04-18T22:23:57.198 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Stddev Latency(s): 0.0164397 2024-04-18T22:23:57.198 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Max latency(s): 1.06735 2024-04-18T22:23:57.198 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Min latency(s): 0.00133878 2024-04-18T22:23:57.204 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Cleaning up (deleting benchmark objects) 2024-04-18T22:23:57.254 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:57.253+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:57.355 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:57.356 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:57.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:57.354+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:57.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:57.456 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:57.454+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:57.524 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:57.525 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:57.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:57.590 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:57.589+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:57.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~23s 2024-04-18T22:23:57.658 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:57.658+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:57.701 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:23:57.701 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:23:57.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:57.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:23:57.762 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:57.761+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:57.860 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:57.858+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:57.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:23:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:23:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:23:57.913 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:23:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:23:57.917 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:23:57.918 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:23:57.922 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:57.922 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:57.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.805267009000005, 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:57.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:57.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.929 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.805134516999999, 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:57.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:57.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:57.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:57.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.939 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.805062930999995, 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:57.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:57.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:57.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.804934677999995, 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:57.948 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.955 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.804825351000005, 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:57.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:57.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:57.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.804569013999995, 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:57.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:57.966 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.804481527999997, 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:57.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.804403542000003, 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:57.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.122116460000001, 2024-04-18T22:23:57.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:57.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:57.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.120460080999997, 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:57.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:57.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.989 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.120335363999999, 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.990 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:57.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:57.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.118555706000002, 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:57.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:57.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.118073035000002, 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:58.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.117833959999999, 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.007 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:58.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:58.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.117713189, 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.013 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:58.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.018 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.114823776999998, 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.114701957999998, 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:58.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.114591509, 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:58.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.039 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.114522041000001, 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:58.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:58.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.045 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 52.114464900000002, 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.046 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:58.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:58.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:58.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:58.051 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.052 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.052 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.052 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:58.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:23:58.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:58.055 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:58.049+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:58.153 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:58.150+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:58.206 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:58.206 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:58.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:130457 5.2 e46/23 5:4f28fc76:::benchmark_data_smithi032_22800_object2343:head v 46'8195, mlcod=46'8193)", 2024-04-18T22:23:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:58.202140+0000", 2024-04-18T22:23:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0044463660000000002, 2024-04-18T22:23:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0044632079999999998, 2024-04-18T22:23:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.208 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:58.208 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202140+0000", 2024-04-18T22:23:58.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202140+0000", 2024-04-18T22:23:58.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202141+0000", 2024-04-18T22:23:58.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4899999999999995e-07 2024-04-18T22:23:58.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202146+0000", 2024-04-18T22:23:58.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1780000000000002e-06 2024-04-18T22:23:58.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202156+0000", 2024-04-18T22:23:58.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7040000000000006e-06 2024-04-18T22:23:58.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202162+0000", 2024-04-18T22:23:58.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7560000000000002e-06 2024-04-18T22:23:58.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202214+0000", 2024-04-18T22:23:58.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1430000000000001e-05 2024-04-18T22:23:58.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202233+0000", 2024-04-18T22:23:58.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9069e-05 2024-04-18T22:23:58.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.216 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:130461 5.1 e46/23 5:898eb047:::benchmark_data_smithi032_22800_object2347:head v 46'8128, mlcod=46'8125)", 2024-04-18T22:23:58.216 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:58.202411+0000", 2024-04-18T22:23:58.216 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0041755960000000002, 2024-04-18T22:23:58.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042306549999999998, 2024-04-18T22:23:58.216 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.217 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.217 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:58.217 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202411+0000", 2024-04-18T22:23:58.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202411+0000", 2024-04-18T22:23:58.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202412+0000", 2024-04-18T22:23:58.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7899999999999998e-07 2024-04-18T22:23:58.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202417+0000", 2024-04-18T22:23:58.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1590000000000001e-06 2024-04-18T22:23:58.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202428+0000", 2024-04-18T22:23:58.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1627000000000001e-05 2024-04-18T22:23:58.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202435+0000", 2024-04-18T22:23:58.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0720000000000002e-06 2024-04-18T22:23:58.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202487+0000", 2024-04-18T22:23:58.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:23:58.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.202510+0000", 2024-04-18T22:23:58.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2589999999999999e-05 2024-04-18T22:23:58.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.224 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.225 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:130464 5.d 5:bc1d95c4:::benchmark_data_smithi032_22800_object2350:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.225 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:58.204431+0000", 2024-04-18T22:23:58.225 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0021552229999999999, 2024-04-18T22:23:58.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022452679999999999, 2024-04-18T22:23:58.226 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.226 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.226 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:58.226 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:58.226 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:58.226 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.227 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 130464 2024-04-18T22:23:58.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.227 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204431+0000", 2024-04-18T22:23:58.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204431+0000", 2024-04-18T22:23:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204432+0000", 2024-04-18T22:23:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.187e-06 2024-04-18T22:23:58.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204438+0000", 2024-04-18T22:23:58.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2950000000000001e-06 2024-04-18T22:23:58.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204451+0000", 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204459+0000", 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3049999999999993e-06 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204509+0000", 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9360000000000002e-05 2024-04-18T22:23:58.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204590+0000", 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1814e-05 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:130463 5.2 e46/23 5:4c28fc1b:::benchmark_data_smithi032_22800_object2349:head v 46'8196, mlcod=46'8193)", 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:58.204767+0000", 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018194039999999999, 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001946136, 2024-04-18T22:23:58.234 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204767+0000", 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204767+0000", 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204768+0000", 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3799999999999997e-07 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204774+0000", 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.789e-06 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204786+0000", 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2435999999999999e-05 2024-04-18T22:23:58.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204793+0000", 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1439999999999997e-06 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204859+0000", 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6032999999999996e-05 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.204887+0000", 2024-04-18T22:23:58.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7818999999999999e-05 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:130467 5.c e46/19 5:35be417e:::benchmark_data_smithi032_22800_object2353:head v 46'8160, mlcod=46'8158)", 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:58.206331+0000", 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00025526200000000002, 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041427400000000001, 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:58.238 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206331+0000", 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206331+0000", 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206332+0000", 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8199999999999999e-07 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206338+0000", 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5949999999999998e-06 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206350+0000", 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2887e-05 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206358+0000", 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.255e-06 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206397+0000", 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9376999999999999e-05 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:58.206408+0000", 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1396999999999999e-05 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 5 2024-04-18T22:23:58.242 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:58.243 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:58.254 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:58.253+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:58.355 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:58.354+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:58.441 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:58.441 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:58.441 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:58.442 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:58.442 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:58.456 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:58.454+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:58.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:58.557 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:58.554+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535639532000005, 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:23:58.652 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:23:58.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:23:58.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535570934000006, 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:23:58.658 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:23:58.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:23:58.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:58.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535475087999998, 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:58.667 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.668 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:23:58.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.670 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.671 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.672 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535418206000003, 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.673 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:23:58.674 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.675 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.676 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:23:58.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535358533999997, 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.679 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.680 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.681 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:23:58.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535300571999997, 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.685 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.686 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:23:58.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.688 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:23:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:23:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.689 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:23:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:23:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:23:58.690 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.691 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:23:58.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.693 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:23:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:23:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.694 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:23:58.695 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:23:58.695 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.695 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.695 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535080879999995, 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.696 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.697 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:23:58.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:23:58.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.700 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.701 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.535023950999999, 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.702 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:23:58.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:23:58.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.534928065000003, 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:23:58.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.710 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:23:58.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.534865737000004, 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:58.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:23:58.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:23:58.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:58.719 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.534793258999997, 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:23:58.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.724 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.534640791000001, 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:23:58.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:23:58.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:23:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:23:58.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.534562068, 2024-04-18T22:23:58.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:23:58.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:23:58.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.533041212000001, 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.737 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.738 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.739 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.740 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.741 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.742 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 86.532942860000006, 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.743 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:23:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:23:58.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 52.851220906999998, 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.747 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.748 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:23:58.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:23:58.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 52.850566694000001, 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.752 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:23:58.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:23:58.756 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 52.850459999000002, 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.760 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 52.848594089999999, 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.761 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:23:58.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 52.847928766999999, 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:58.766 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:23:58.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:58.770 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:58.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:23:58.773 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:58.769+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:58.873 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:58.869+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:58.927 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:58.927 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:58.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.927 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132266 5.a 5:52f3d8ed:::benchmark_data_smithi032_22800_object4152:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.927 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.921264+0000", 2024-04-18T22:23:58.927 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0051652440000000003, 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051945469999999999, 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132266 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.921264+0000", 2024-04-18T22:23:58.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.921264+0000", 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.921265+0000", 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9500000000000002e-07 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.921268+0000", 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4510000000000001e-06 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.921276+0000", 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1690000000000003e-06 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.921280+0000", 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.775e-06 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.922828+0000", 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015472140000000001 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.922904+0000", 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6371000000000005e-05 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.931 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132267 5.1 5:81645695:::benchmark_data_smithi032_22800_object4153:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.923225+0000", 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032046409999999998, 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033255289999999998, 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.932 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132267 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923225+0000", 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923225+0000", 2024-04-18T22:23:58.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923225+0000", 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923229+0000", 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3500000000000001e-06 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923237+0000", 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4370000000000007e-06 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923242+0000", 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5959999999999997e-06 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923287+0000", 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5407000000000001e-05 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923391+0000", 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923503+0000", 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112495 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.936 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132270 5.1 5:8bb7049f:::benchmark_data_smithi032_22800_object4156:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.923263+0000", 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003166416, 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033826020000000002, 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132270 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.937 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923263+0000", 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923263+0000", 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923264+0000", 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923266+0000", 2024-04-18T22:23:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7300000000000001e-06 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923274+0000", 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6869999999999994e-06 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923281+0000", 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1239999999999999e-06 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.924900+0000", 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001618499 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925057+0000", 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157727 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925191+0000", 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013305899999999999 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132271 5.c 5:3f3efda3:::benchmark_data_smithi032_22800_object4157:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.923294+0000", 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0031358720000000001, 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034410119999999998, 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132271 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923294+0000", 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923294+0000", 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923294+0000", 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2799999999999996e-07 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923297+0000", 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5320000000000001e-06 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923310+0000", 2024-04-18T22:23:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3247e-05 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923317+0000", 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2760000000000003e-06 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923368+0000", 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0834999999999997e-05 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923502+0000", 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000134239 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923617+0000", 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114975 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132272 5.a 5:58335cd5:::benchmark_data_smithi032_22800_object4158:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.923943+0000", 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0024868519999999999, 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028807400000000001, 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132272 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923943+0000", 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923943+0000", 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923943+0000", 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1900000000000002e-07 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923948+0000", 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6260000000000003e-06 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923959+0000", 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1077000000000001e-05 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.923966+0000", 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6200000000000001e-06 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.924389+0000", 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00042346300000000001 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.924468+0000", 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8658000000000005e-05 2024-04-18T22:23:58.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132273 5.3 5:c7a269c5:::benchmark_data_smithi032_22800_object4159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.925175+0000", 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0012544710000000001, 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001729167, 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.951 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132273 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925175+0000", 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925175+0000", 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925176+0000", 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7199999999999999e-07 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925179+0000", 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6430000000000001e-06 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925190+0000", 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0437e-05 2024-04-18T22:23:58.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925197+0000", 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8709999999999999e-06 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925249+0000", 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2247999999999998e-05 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925366+0000", 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117068 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925439+0000", 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2726999999999997e-05 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132274 5.e 5:728981c1:::benchmark_data_smithi032_22800_object4160:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.955 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.925708+0000", 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00072170199999999996, 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012841599999999999, 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132274 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.956 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925708+0000", 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925708+0000", 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925709+0000", 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9100000000000003e-07 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925713+0000", 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0860000000000004e-06 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925723+0000", 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.047e-05 2024-04-18T22:23:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925730+0000", 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3929999999999996e-06 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925782+0000", 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2491000000000003e-05 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.925855+0000", 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2424999999999999e-05 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132275 5.e 5.940a5e5e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.926119+0000", 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00031078399999999998, 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00095749500000000005, 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132275 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926119+0000", 2024-04-18T22:23:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926119+0000", 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926119+0000", 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8199999999999999e-07 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926123+0000", 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.191e-06 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926132+0000", 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1689999999999998e-06 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926137+0000", 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.604e-06 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:132261 5.6 e46/19)", 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.926348+0000", 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.1737999999999996e-05, 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000790068, 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926348+0000", 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926348+0000", 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926348+0000", 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9200000000000001e-07 2024-04-18T22:23:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926352+0000", 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7139999999999999e-06 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926361+0000", 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4319999999999999e-06 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926365+0000", 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8999999999999997e-06 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:132262 5.6 e46/19)", 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.926437+0000", 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999928, 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00075671799999999997, 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926437+0000", 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926437+0000", 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926437+0000", 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8200000000000001e-07 2024-04-18T22:23:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926439+0000", 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2110000000000001e-06 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926445+0000", 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5260000000000001e-06 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926448+0000", 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6759999999999999e-06 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132276 5.a 5:5a9b099f:::benchmark_data_smithi032_22800_object4162:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.926946+0000", 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9994836, 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031051700000000001, 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:58.970 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132276 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926946+0000", 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926946+0000", 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926946+0000", 2024-04-18T22:23:58.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.37e-07 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926949+0000", 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.959e-06 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926956+0000", 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926960+0000", 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2629999999999997e-06 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927004+0000", 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3807000000000003e-05 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927137+0000", 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013332700000000001 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.974 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132277 5.e 5.9ad05f2e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.926971+0000", 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9994588, 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036658699999999998, 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.975 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132277 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926971+0000", 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926971+0000", 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926971+0000", 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8099999999999999e-07 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926973+0000", 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.68e-06 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926976+0000", 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7160000000000001e-06 2024-04-18T22:23:58.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.926980+0000", 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0460000000000001e-06 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132278 5.4 5:2842d9c0:::benchmark_data_smithi032_22800_object4164:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.927143+0000", 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9992862, 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026330700000000001, 2024-04-18T22:23:58.978 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132278 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927143+0000", 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927143+0000", 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927144+0000", 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6699999999999999e-07 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927146+0000", 2024-04-18T22:23:58.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2139999999999999e-06 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927152+0000", 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0970000000000002e-06 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927156+0000", 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6370000000000001e-06 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927188+0000", 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2214e-05 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.927263+0000", 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5457999999999995e-05 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132261 5.6 5:669fd571:::benchmark_data_smithi032_22800_object4147:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.918019+0000", 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.008410308, 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0094697240000000005, 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.983 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132261 2024-04-18T22:23:58.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.984 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918019+0000", 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918019+0000", 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918020+0000", 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2e-07 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918022+0000", 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3479999999999998e-06 2024-04-18T22:23:58.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918028+0000", 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.648e-06 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918032+0000", 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4220000000000002e-06 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918372+0000", 2024-04-18T22:23:58.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000340136 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918480+0000", 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010738699999999999 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918588+0000", 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010885799999999999 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.922002+0000", 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034132839999999999 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132262 5.6 5:69d75257:::benchmark_data_smithi032_22800_object4148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.918900+0000", 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0075298869999999999, 2024-04-18T22:23:58.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0086885439999999994, 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132262 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918900+0000", 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918900+0000", 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918900+0000", 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7899999999999998e-07 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918904+0000", 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0539999999999996e-06 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918915+0000", 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0718e-05 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.918922+0000", 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9040000000000003e-06 2024-04-18T22:23:58.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920236+0000", 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001314464 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920352+0000", 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115518 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920475+0000", 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012319100000000001 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.922459+0000", 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019835170000000002 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.994 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:132264 5.7 5:edde16cc:::benchmark_data_smithi032_22800_object4150:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:58.920574+0000", 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0058553659999999999, 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0071106379999999999, 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:58.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 132264 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920574+0000", 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920574+0000", 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920575+0000", 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7499999999999999e-07 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920579+0000", 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.233e-06 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920589+0000", 2024-04-18T22:23:58.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8109999999999997e-06 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920596+0000", 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6850000000000004e-06 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920652+0000", 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6440000000000002e-05 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920783+0000", 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130727 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:58.920890+0000", 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106901 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 16 2024-04-18T22:23:58.999 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:59.001 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:58.998+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:59.002 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:23:59.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:23:59.102 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:59.101+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:59.193 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:59.193 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:23:59.193 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:23:59.193 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:23:59.193 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:59.202 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:59.201+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:59.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:23:59.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:59.302+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:59.380 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [1] live_osds: [0, 2] 2024-04-18T22:23:59.380 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:23:59.380 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-18T22:23:59.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-04-18T22:23:59.403 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:59.401+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.297601963000005, 2024-04-18T22:23:59.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.416 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.418 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:23:59.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:59.422 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.297469470999999, 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:23:59.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:23:59.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.429 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.297397884999995, 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:59.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:23:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:23:59.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:23:59.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.297269631999995, 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:23:59.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.443 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.297160305000006, 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:23:59.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:23:59.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:23:59.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:23:59.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.296903967999995, 2024-04-18T22:23:59.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:23:59.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:23:59.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:23:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.296816481999997, 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:59.457 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:23:59.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:23:59.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:23:59.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 87.296738496000003, 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:23:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:23:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.614451414000001, 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.470 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:23:59.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:23:59.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:23:59.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.612795034999998, 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:23:59.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.483 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.612670317999999, 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:23:59.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:23:59.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:23:59.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.610890660000003, 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:23:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.610407989000002, 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.503 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.610168913999999, 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:23:59.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:23:59.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:23:59.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.610048143, 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:23:59.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.607158730999998, 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:23:59.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:23:59.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:23:59.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.523 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.607036911999998, 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:23:59.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:23:59.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:23:59.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.606926463000001, 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:23:59.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:23:59.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:23:59.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.606856995000001, 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:23:59.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:23:59.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:59.543 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 53.606799854000002, 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:23:59.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:23:59.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:23:59.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:23:59.549 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:23:59.550 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:23:59.551 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:23:59.552 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:59.549+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:59.651 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:23:59.649+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:23:59.689 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 242 (0%) 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:avg 121 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 38.0526 -> 38.0526 (expected baseline 7.77817) 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.2 with 81 -> 81 pgs (0.669421 -> 0.669421 * mean) 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.0 with 85 -> 85 pgs (0.702479 -> 0.702479 * mean) 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:oload 1000 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 0.05 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0046 2024-04-18T22:23:59.690 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0458 2024-04-18T22:23:59.691 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-04-18T22:23:59.701 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:23:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.701 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:134120 5.6 e47/19 5:61dde0da:::benchmark_data_smithi032_22800_object6006:head v 47'8337, mlcod=47'8334)", 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.696908+0000", 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0051729430000000002, 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0051965800000000001, 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696908+0000", 2024-04-18T22:23:59.702 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696908+0000", 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696908+0000", 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7e-07 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.703 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696915+0000", 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.046e-06 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696922+0000", 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.165e-06 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.704 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696928+0000", 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7239999999999997e-06 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696973+0000", 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5076e-05 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696984+0000", 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1029999999999999e-05 2024-04-18T22:23:59.705 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:134115 5.3 e47/19 5:cf3f462c:::benchmark_data_smithi032_22800_object6001:head v 47'8529, mlcod=47'8526)", 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.696940+0000", 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0051405319999999997, 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0052358229999999997, 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.706 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696940+0000", 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696940+0000", 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.707 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696940+0000", 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6199999999999999e-07 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696944+0000", 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7900000000000001e-06 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696955+0000", 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0922e-05 2024-04-18T22:23:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696960+0000", 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.707e-06 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.696994+0000", 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4057000000000003e-05 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.709 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.697005+0000", 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1299e-05 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:134124 5.9 5:9ab27144:::benchmark_data_smithi032_22800_object6010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.698504+0000", 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0035764439999999998, 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0037348519999999999, 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.710 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 134124 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698504+0000", 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698504+0000", 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698505+0000", 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.164e-06 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698513+0000", 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9859999999999994e-06 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698529+0000", 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5947000000000002e-05 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698539+0000", 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0183999999999999e-05 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698584+0000", 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5031000000000003e-05 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.698680+0000", 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5478999999999998e-05 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:134125 5.3 e47/19 5:c1cffb73:::benchmark_data_smithi032_22800_object6011:head v 47'8530, mlcod=47'8526)", 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.699052+0000", 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0030288329999999999, 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032546139999999999, 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699052+0000", 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699052+0000", 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699052+0000", 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4000000000000001e-07 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699061+0000", 2024-04-18T22:23:59.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6929999999999996e-06 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699073+0000", 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2013e-05 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699080+0000", 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6579999999999996e-06 2024-04-18T22:23:59.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699117+0000", 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7085000000000002e-05 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.699129+0000", 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2306e-05 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:134127 5.3 e47/19 5:c6fa4715:::benchmark_data_smithi032_22800_object6013:head v 47'8531, mlcod=47'8526)", 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.701175+0000", 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00090561899999999998, 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011932959999999999, 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701175+0000", 2024-04-18T22:23:59.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701175+0000", 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701176+0000", 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7000000000000004e-07 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701184+0000", 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4519999999999997e-06 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701197+0000", 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2839e-05 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701204+0000", 2024-04-18T22:23:59.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9489999999999999e-06 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701248+0000", 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4286999999999998e-05 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701268+0000", 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9534999999999998e-05 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.722 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:134130 5.6 e47/19 5:669a307f:::benchmark_data_smithi032_22800_object6016:head v 47'8338, mlcod=47'8335)", 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.701301+0000", 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00077961199999999995, 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011074909999999999, 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701301+0000", 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701301+0000", 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701302+0000", 2024-04-18T22:23:59.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6599999999999996e-07 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701309+0000", 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6320000000000008e-06 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701323+0000", 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3522e-05 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701329+0000", 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1990000000000003e-06 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701367+0000", 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7783000000000001e-05 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.701379+0000", 2024-04-18T22:23:59.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2108e-05 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:134132 5.7 e47/23 5:e3d8529e:::benchmark_data_smithi032_22800_object6018:head v 47'8450, mlcod=47'8448)", 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:59.702154+0000", 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.999927, 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028588399999999998, 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:23:59.727 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702154+0000", 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702154+0000", 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702154+0000", 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3799999999999997e-07 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702162+0000", 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5320000000000001e-06 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702172+0000", 2024-04-18T22:23:59.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8009999999999998e-06 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702176+0000", 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.729e-06 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702250+0000", 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3323000000000003e-05 2024-04-18T22:23:59.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:59.702263+0000", 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3471e-05 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 7 2024-04-18T22:23:59.731 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:59.733 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-04-18T22:23:59.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-04-18T22:23:59.734 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:23:59.779 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:59.778+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:59.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:59.850+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:23:59.871 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:23:59.871 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:23:59.871 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:23:59.871 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:23:59.871 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:23:59.891 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:23:59.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:23:59.983 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:23:59.982+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:00.043 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 242 (0%) 2024-04-18T22:24:00.043 INFO:teuthology.orchestra.run.smithi032.stdout:avg 121 2024-04-18T22:24:00.043 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 38.0526 -> 38.0526 (expected baseline 7.77817) 2024-04-18T22:24:00.043 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.2 with 81 -> 81 pgs (0.669421 -> 0.669421 * mean) 2024-04-18T22:24:00.043 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.0 with 85 -> 85 pgs (0.702479 -> 0.702479 * mean) 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stdout:oload 1000 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 1 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0046 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0458 2024-04-18T22:24:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-04-18T22:24:00.054 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.053+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.066 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-04-18T22:24:00.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-04-18T22:24:00.079 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:00.090 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:00.090 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:00.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:00.090 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:00.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.974015539999996, 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973946941999998, 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.096 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.101 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973851096000004, 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.104 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973794213999994, 2024-04-18T22:24:00.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:00.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:00.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:00.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.113 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973734542000003, 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:00.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973676580000003, 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.120 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:00.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:00.126 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973456888000001, 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:00.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:00.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973399959000005, 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.133 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:24:00.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.139 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973304072999994, 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:00.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:24:00.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.145 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973241744999996, 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:24:00.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:24:00.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:24:00.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.152 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973169267000003, 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:00.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:00.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.158 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.973016799000007, 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:00.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:00.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:00.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.972938076000005, 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:24:00.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:24:00.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:24:00.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.971417220000006, 2024-04-18T22:24:00.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:00.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:00.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:00.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.177 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 87.971318867999997, 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:00.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:00.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:00.182 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 54.289596914999997, 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:00.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:00.187 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 54.288942702, 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:00.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:00.192 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 54.288836007, 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:00.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:00.197 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 54.286970097999998, 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:00.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:00.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 54.286304774999998, 2024-04-18T22:24:00.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:00.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:00.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:00.207 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:00.208 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:00.209 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:00.210 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:00.206+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:00.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:00.211 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:00.313 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.305+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.362 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:00.362 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:00.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:00.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:00.410+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:00.512 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.509+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.572 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:00.573 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:00.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:00.635 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.633+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.714 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.709+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.782 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:00.783 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:00.802 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:00.842 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.841+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.916 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:00.913+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:00.951 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:00.952 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:00.971 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:00.971 INFO:teuthology.orchestra.run.smithi032.stdout:moved 33 / 242 (13.6364%) 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:avg 121 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 38.0526 -> 51.4296 (expected baseline 7.77817) 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.2 with 81 -> 98 pgs (0.669421 -> 0.809917 * mean) 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.0 with 85 -> 52 pgs (0.702479 -> 0.429752 * mean) 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:oload 110 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 3 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0046 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0050 2024-04-18T22:24:00.972 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 weight 1.0000 -> 0.6897 2024-04-18T22:24:00.996 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-04-18T22:24:00.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-04-18T22:24:01.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:24:01.019 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.018+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.090 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.090 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.104 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:01.147 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.146+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.220 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:01.216+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:01.271 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.271 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.286 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:01.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-18T22:24:01.305+0000 7f085e731640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-04-18T22:24:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:moved 3 / 242 (1.23967%) 2024-04-18T22:24:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:avg 121 2024-04-18T22:24:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 51.4296 -> 53.2353 (expected baseline 7.77817) 2024-04-18T22:24:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.0 with 52 -> 49 pgs (0.429752 -> 0.404959 * mean) 2024-04-18T22:24:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.2 with 98 -> 99 pgs (0.809917 -> 0.818182 * mean) 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout:oload 110 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 0.05 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0046 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0051 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 weight 0.6897 -> 0.6397 2024-04-18T22:24:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-04-18T22:24:01.320 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.318+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.329 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-04-18T22:24:01.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-04-18T22:24:01.409 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.410 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.418+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:01.522 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.522+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.528 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.528 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:01.623 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.622+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 242 (0%) 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout:avg 121 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 51.4296 -> 51.4296 (expected baseline 7.77817) 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.0 with 52 -> 52 pgs (0.429752 -> 0.429752 * mean) 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.2 with 98 -> 98 pgs (0.809917 -> 0.809917 * mean) 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:24:01.634 INFO:teuthology.orchestra.run.smithi032.stdout:oload 1000 2024-04-18T22:24:01.635 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 3 2024-04-18T22:24:01.635 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-04-18T22:24:01.635 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0046 2024-04-18T22:24:01.635 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0461 2024-04-18T22:24:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-04-18T22:24:01.647 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.647 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.660 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:01.724 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.722+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.791 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.791 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.805 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:01.851 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:01.850+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:01.925 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:01.924+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:24:01.974 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:01.974 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:01.992 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:02.026 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.024+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.107 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:02.107 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:02.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:02.127 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:02.126+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:02.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:02.228 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.224+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.247 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:02.247 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:02.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:02.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:02.326+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:02.371 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:02.372 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:02.372 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:02.393 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:02.397 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:02.397 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:02.422 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:02.422 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:02.423 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:02.423 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:02.423 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:02.429 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.424+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:02.529 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.524+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:24:02.597 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478794687999994, 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037320755999999997, 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115597+0000", 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115600+0000", 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0259999999999999e-06 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115614+0000", 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4382999999999999e-05 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115647+0000", 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3149999999999999e-05 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115673+0000", 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5933999999999999e-05 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115797+0000", 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012428600000000001 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115807+0000", 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9860000000000006e-06 2024-04-18T22:24:02.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133329+0000", 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017521666000000002 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133653+0000", 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032409799999999999 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133974+0000", 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032088499999999997 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140275+0000", 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063006410000000001 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152707+0000", 2024-04-18T22:24:02.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012432591999999999 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152801+0000", 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3659000000000006e-05 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.152917+0000", 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011645 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.604 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478662196000002, 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.037572080000000001, 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115729+0000", 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115731+0000", 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.556e-06 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115739+0000", 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6230000000000003e-06 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115757+0000", 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8091e-05 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115773+0000", 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5089000000000001e-05 2024-04-18T22:24:02.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115952+0000", 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000179261 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115977+0000", 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4927999999999999e-05 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135495+0000", 2024-04-18T22:24:02.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019518333999999998 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135710+0000", 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021527199999999999 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136077+0000", 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00036705700000000001 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140309+0000", 2024-04-18T22:24:02.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004231645 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153005+0000", 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012695970000000001 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153141+0000", 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013562099999999999 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153301+0000", 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016063300000000001 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:18 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478590609999998, 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038591294999999998, 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115801+0000", 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115802+0000", 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3599999999999999e-06 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115810+0000", 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115827+0000", 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7431e-05 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6049000000000002e-05 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116034+0000", 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019123299999999999 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116048+0000", 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3319000000000001e-05 2024-04-18T22:24:02.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133597+0000", 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.017549914 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133877+0000", 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000279802 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134195+0000", 2024-04-18T22:24:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000318098 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139560+0000", 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053650900000000003 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154107+0000", 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014546524 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154284+0000", 2024-04-18T22:24:02.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017671399999999999 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154392+0000", 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010839400000000001 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:36 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478462356999998, 2024-04-18T22:24:02.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038918539000000002, 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115929+0000", 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115931+0000", 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5060000000000001e-06 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115939+0000", 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0229999999999992e-06 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115958+0000", 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9607999999999999e-05 2024-04-18T22:24:02.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.115975+0000", 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7054999999999998e-05 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016619900000000001 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116152+0000", 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0069e-05 2024-04-18T22:24:02.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135388+0000", 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.019236626 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135614+0000", 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000225403 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.135893+0000", 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027953899999999998 2024-04-18T22:24:02.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140295+0000", 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044018879999999996 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154516+0000", 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014221515000000001 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154749+0000", 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023264499999999999 2024-04-18T22:24:02.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154848+0000", 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8462999999999996e-05 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:19 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478353029999994, 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039373075, 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116038+0000", 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116040+0000", 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3230000000000001e-06 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116096+0000", 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6307999999999997e-05 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116122+0000", 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6344999999999999e-05 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116141+0000", 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8201000000000001e-05 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116322+0000", 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018134499999999999 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116333+0000", 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1122e-05 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.136710+0000", 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020377309 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138170+0000", 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001459215 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138497+0000", 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000327418 2024-04-18T22:24:02.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.140325+0000", 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00182803 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155218+0000", 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014892766 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155334+0000", 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011585 2024-04-18T22:24:02.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155411+0000", 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7843000000000007e-05 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:28 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478096692999998, 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.036775664, 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116295+0000", 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116296+0000", 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4670000000000001e-06 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116307+0000", 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0749e-05 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116325+0000", 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7844999999999999e-05 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116360+0000", 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5110000000000001e-05 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119759+0000", 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003398739 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.119774+0000", 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5319e-05 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.133764+0000", 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.013989812000000001 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.134123+0000", 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035879899999999998 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.153070+0000", 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018947823999999999 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.633 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:30 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.478009206999999, 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039199233, 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116382+0000", 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116384+0000", 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.53e-06 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116390+0000", 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2530000000000001e-06 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116410+0000", 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0341000000000001e-05 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116424+0000", 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3552e-05 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116691+0000", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267383 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting_for_map not empty", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116702+0000", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0993e-05 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139386+0000", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.022683430000000001 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.139716+0000", 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033059799999999998 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.155581+0000", 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.015865153 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(mds.0.3:33 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:24:02.638 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 90.477931221000006, 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.038120135999999999, 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116460+0000", 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116462+0000", 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.483e-06 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116468+0000", 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.127e-06 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116488+0000", 2024-04-18T22:24:02.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0228e-05 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116502+0000", 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4102e-05 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116633+0000", 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013115199999999999 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op must wait for map", 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.116647+0000", 2024-04-18T22:24:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3961999999999999e-05 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.137983+0000", 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.021335592 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.138148+0000", 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016523199999999999 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:22:32.154580+0000", 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.016432258000000002 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.795644138999997, 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.648 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.79398776, 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.793863043000002, 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.659 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.792083384999998, 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:02.665 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.791600713999998, 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:02.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:02.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.791361639000002, 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:02.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.791240868000003, 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:02.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.788351456000001, 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:02.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:02.689 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.788229637000001, 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.690 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.695 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.788119188000003, 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:02.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.788049719999997, 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:02.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 56.787992578999997, 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:02.707 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:02.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:02.712 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:02.714 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:02.714 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.712+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.814 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.812+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.847 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:02.847 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:02.847 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-04-18T22:24:02.847 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:02.861 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:02.916 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:02.912+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:02.991 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:02.991 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:02.991 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:02.991 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:02.991 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:03.012 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:03.059 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:03.058+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:03.117 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:03.114+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:03.158 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:03.158 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.041555239999994, 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.159 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:03.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:03.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:03.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.164 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.041486641999995, 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:03.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.041390796000002, 2024-04-18T22:24:03.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:03.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:03.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.041333914000006, 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:03.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.041274242, 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.182 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:03.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:03.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:03.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.04121628, 2024-04-18T22:24:03.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:03.192 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040996587999999, 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:03.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:03.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040939659000003, 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.197 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040843773000006, 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:24:03.202 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:03.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040781444999993, 2024-04-18T22:24:03.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:24:03.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040708967, 2024-04-18T22:24:03.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:03.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:03.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040556499000004, 2024-04-18T22:24:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.040477776000003, 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:24:03.222 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:24:03.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:24:03.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.038956920000004, 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:03.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:03.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 91.038858567999995, 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:03.233 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.357136615000002, 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.237 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.356482401999997, 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:03.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:03.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.356375706999998, 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:03.246 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:03.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:03.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:03.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.354509798000002, 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:03.250 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:03.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.254 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 57.353844475000002, 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:03.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:03.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:03.259 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:03.260 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:03.256+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:03.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:03.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:03.358+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:03.428 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:03.428 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:03.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.428 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138005 5.3 5:c58b261a:::benchmark_data_smithi032_22800_object9891:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.428 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.424565+0000", 2024-04-18T22:24:03.428 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0036431459999999999, 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036687270000000001, 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138005 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.429 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424565+0000", 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424565+0000", 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424566+0000", 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7400000000000003e-07 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424569+0000", 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3850000000000002e-06 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424578+0000", 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3040000000000005e-06 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424584+0000", 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2559999999999999e-06 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.426280+0000", 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001696253 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.426398+0000", 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117478 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.426505+0000", 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107706 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138006 5.c 5:3623ae82:::benchmark_data_smithi032_22800_object9892:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.424604+0000", 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003604309, 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037127100000000001, 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138006 2024-04-18T22:24:03.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424604+0000", 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424604+0000", 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424605+0000", 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8700000000000005e-07 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424611+0000", 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3130000000000004e-06 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424620+0000", 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7190000000000007e-06 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424626+0000", 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1480000000000003e-06 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424773+0000", 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014736600000000001 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.425030+0000", 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00025644399999999999 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.425153+0000", 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012329800000000001 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428160+0000", 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030067129999999998 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138009 5.3 5:cf538245:::benchmark_data_smithi032_22800_object9895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.424690+0000", 2024-04-18T22:24:03.437 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0035186039999999998, 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037089860000000001, 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138009 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424690+0000", 2024-04-18T22:24:03.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424690+0000", 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424691+0000", 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6599999999999995e-07 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424695+0000", 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9589999999999997e-06 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424704+0000", 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.73e-06 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424711+0000", 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7159999999999997e-06 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427970+0000", 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032587079999999999 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428046+0000", 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6311999999999997e-05 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428136+0000", 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0233000000000002e-05 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138010 5.4 5:2f901395:::benchmark_data_smithi032_22800_object9896:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.424850+0000", 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0033583129999999999, 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036213719999999999, 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138010 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424850+0000", 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424850+0000", 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424851+0000", 2024-04-18T22:24:03.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9800000000000002e-07 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424862+0000", 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0743e-05 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424872+0000", 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0421999999999999e-05 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424879+0000", 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9110000000000004e-06 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.425787+0000", 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00090742399999999999 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.425926+0000", 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000139705 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.445 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138012 5.0 5:0d261229:::benchmark_data_smithi032_22800_object9898:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.427083+0000", 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0011253400000000001, 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014549560000000001, 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138012 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427083+0000", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427083+0000", 2024-04-18T22:24:03.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427084+0000", 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.093e-06 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427090+0000", 2024-04-18T22:24:03.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2689999999999999e-06 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427105+0000", 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5302e-05 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427113+0000", 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3690000000000007e-06 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427907+0000", 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00079385199999999995 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428072+0000", 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000164962 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428206+0000", 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013412299999999999 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138013 5.c 5:37cdc6d3:::benchmark_data_smithi032_22800_object9899:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.428275+0000", 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999337, 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033701500000000003, 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138013 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428275+0000", 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428275+0000", 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428275+0000", 2024-04-18T22:24:03.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7800000000000001e-07 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428284+0000", 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5029999999999997e-06 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428292+0000", 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8879999999999997e-06 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428297+0000", 2024-04-18T22:24:03.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0520000000000001e-06 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428354+0000", 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7006999999999998e-05 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428485+0000", 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013092600000000001 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428563+0000", 2024-04-18T22:24:03.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8628999999999995e-05 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138015 5.e 5:71685eba:::benchmark_data_smithi032_22800_object9901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.428328+0000", 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998803, 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035992799999999997, 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138015 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428328+0000", 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428328+0000", 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428329+0000", 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0699999999999997e-07 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428335+0000", 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.601e-06 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428344+0000", 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6710000000000005e-06 2024-04-18T22:24:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428350+0000", 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4219999999999997e-06 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428458+0000", 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107612 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428562+0000", 2024-04-18T22:24:03.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104274 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138016 5.c 5.de3af02c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.428361+0000", 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998479, 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00039275399999999999, 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.459 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138016 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428361+0000", 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428361+0000", 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428361+0000", 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1600000000000002e-07 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428363+0000", 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8759999999999999e-06 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428367+0000", 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1899999999999997e-06 2024-04-18T22:24:03.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.428370+0000", 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.146e-06 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137997 5.b 5:d829d9dc:::benchmark_data_smithi032_22800_object9883:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.419440+0000", 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0087683239999999992, 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093684149999999997, 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137997 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419440+0000", 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419440+0000", 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419441+0000", 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.71e-07 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419446+0000", 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6190000000000002e-06 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419456+0000", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0627e-05 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419463+0000", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9809999999999997e-06 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.419876+0000", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00041243600000000002 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.420002+0000", 2024-04-18T22:24:03.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012647299999999999 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138002 5.3 5:cd1d0714:::benchmark_data_smithi032_22800_object9888:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.424110+0000", 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.004098473, 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047642409999999998, 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.466 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138002 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424110+0000", 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424110+0000", 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424111+0000", 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4600000000000005e-07 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424114+0000", 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4020000000000002e-06 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424123+0000", 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0939999999999997e-06 2024-04-18T22:24:03.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424130+0000", 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4289999999999998e-06 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424199+0000", 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9397e-05 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424294+0000", 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4825000000000001e-05 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424374+0000", 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0513000000000007e-05 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.427952+0000", 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035776829999999999 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:138004 5.8 5:1e7a0279:::benchmark_data_smithi032_22800_object9890:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.424511+0000", 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0036977220000000001, 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044415330000000001, 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 138004 2024-04-18T22:24:03.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424511+0000", 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424511+0000", 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424512+0000", 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.037e-06 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424517+0000", 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0810000000000001e-06 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424538+0000", 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0682999999999998e-05 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424546+0000", 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1659999999999994e-06 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424597+0000", 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1328000000000001e-05 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.424706+0000", 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109287 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.474 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.475 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:03.475 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 11 2024-04-18T22:24:03.475 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:03.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:03.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:03.474+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:03.576 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~29s 2024-04-18T22:24:03.577 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:03.574+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:03.648 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:03.648 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:03.648 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:03.648 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:03.648 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:03.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:03.715 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:03.712+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:03.779 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:03.776+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:03.843 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:03.843 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:03.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:03.843 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.042583587999999, 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.844 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:03.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.040927209000003, 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:03.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:03.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.040802491999997, 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:03.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:03.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.039022834000001, 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.862 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.863 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.864 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:03.866 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.038540163, 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.871 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:03.873 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.038301087999997, 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:03.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:03.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:03.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.879 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.038180316999998, 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:03.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:03.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:03.885 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.035290904999997, 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:03.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:03.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:03.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.891 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.035169086000003, 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:03.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.035058636999999, 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.897 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:03.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:03.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.034989168999999, 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.908 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 58.034932028, 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:03.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:03.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.914 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.80575651699999995, 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.915 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.920 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.80111351799999997, 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.79201578100000003, 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.932 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.78299585599999999, 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:03.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:03.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:03.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:03.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.76698448799999996, 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:03.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.76418106799999996, 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:03.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:03.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:03.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:03.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137384 5.0 5:0d4b5451:::benchmark_data_smithi032_22800_object9270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.71880449999999996, 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.054059477000000002, 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137384 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.954 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122528+0000", 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1629999999999999e-06 2024-04-18T22:24:03.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122533+0000", 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5570000000000003e-06 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122554+0000", 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1222000000000001e-05 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122571+0000", 2024-04-18T22:24:03.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7005000000000001e-05 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156535+0000", 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.033963356 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156833+0000", 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00029849800000000001 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.157060+0000", 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022695300000000001 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175844+0000", 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018784072999999998 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176295+0000", 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00045056399999999998 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176394+0000", 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9667000000000006e-05 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176586+0000", 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019141899999999999 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137394 5.0 5:082322e3:::benchmark_data_smithi032_22800_object9280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:03.959 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.70845476299999999, 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.050257665, 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137394 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:03.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132878+0000", 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6169999999999999e-06 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:03.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132885+0000", 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.126e-06 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132905+0000", 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0369000000000001e-05 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132921+0000", 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172761+0000", 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039840484000000002 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172994+0000", 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023306499999999999 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.173221+0000", 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000226949 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.179433+0000", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062120439999999999 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182890+0000", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034568089999999999 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182980+0000", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9844000000000002e-05 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:03.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.183134+0000", 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015365800000000001 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:03.965 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:03.966 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:03.967 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:03.960+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:03.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:24:04.067 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:04.064+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:04.120 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:04.120 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:139632 5.3 e50/19 5:cc6f89f2:::benchmark_data_smithi032_22800_object11518:head v 50'8904, mlcod=50'8902)", 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:04.120197+0000", 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0013384670000000001, 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0013570279999999999, 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120197+0000", 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120197+0000", 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120197+0000", 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5700000000000002e-07 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120203+0000", 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5509999999999999e-06 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120212+0000", 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0599999999999997e-06 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120228+0000", 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6543999999999999e-05 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120274+0000", 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5935e-05 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120287+0000", 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2653999999999999e-05 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.124 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:139633 5.f 5:f82b61d9:::benchmark_data_smithi032_22800_object11519:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:04.120769+0000", 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00076633599999999995, 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00082647500000000002, 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 139633 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120769+0000", 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120769+0000", 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120770+0000", 2024-04-18T22:24:04.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.004e-06 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120774+0000", 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0380000000000001e-06 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120785+0000", 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1592e-05 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120793+0000", 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2910000000000002e-06 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120834+0000", 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1294000000000001e-05 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:04.120947+0000", 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112799 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.128 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:04.129 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 2 2024-04-18T22:24:04.129 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:04.141 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:04.187 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:04.186+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:04.269 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:04.264+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:04.302 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:04.302 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:04.302 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:04.302 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:04.302 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:04.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:04.369 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:04.364+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:04.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:04.470 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:04.468+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:04.510 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:04.510 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:04.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:04.510 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.394351865999994, 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:04.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.515 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.394283267999995, 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.516 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:04.517 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.518 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.519 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.520 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.394187422000002, 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:04.521 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:04.523 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.524 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.525 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.526 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.394130540000006, 2024-04-18T22:24:04.527 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:04.529 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.530 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.531 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:04.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.394070868, 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.533 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:04.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:04.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.394012906, 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.538 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:04.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393793213999999, 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.543 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:04.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.546 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.548 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393736285000003, 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:24:04.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.554 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393640399000006, 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393578070999993, 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.560 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393505593, 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.565 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:04.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393353125000004, 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.571 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:04.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:04.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.576 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.393274402000003, 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:04.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.581 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.391753546000004, 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:04.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.584 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.391655193999995, 2024-04-18T22:24:04.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.588 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.589 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.590 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 58.709933241000002, 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.591 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.592 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.593 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.594 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 58.709279027999997, 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.595 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:04.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.597 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:04.598 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 58.709172332999998, 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.599 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:04.600 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:04.601 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:04.602 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 58.707306424000002, 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:04.606 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 58.706641101000002, 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:04.607 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:04.608 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:04.609 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.610 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:04.611 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:04.612 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:04.612 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:04.613 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:04.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:04.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:04.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:04.616 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:04.608+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:04.714 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:04.708+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:04.765 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:04.766 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:04.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:04.815 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:04.814+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:04.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:04.914+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:04.982 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:04.983 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:04.997 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:24:05.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:05.047 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:05.044+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:05.117 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:05.112+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:05.153 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:05.153 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:05.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:05.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:05.218+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:05.318 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:05.316+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:05.322 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:05.322 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:05.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:05.420 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:05.416+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:05.521 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:05.516+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:05.536 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:05.536 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:05.556 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:05.621 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:05.616+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:05.713 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:05.713 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:05.722 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:05.718+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:05.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:05.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:05.822+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:05.891 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:05.891 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:05.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:05.924 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:05.922+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:06.024 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:06.020+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:06.064 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:06.064 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:06.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:06.127 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:06.124+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:06.227 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:06.224+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:06.275 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:06.276 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:06.296 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:06.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:06.342+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:06.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:06.426+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:06.449 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:06.450 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:06.470 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:06.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:06.554+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:06.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:06.629 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:06.626+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:06.654 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [1] live_osds: [0, 2] 2024-04-18T22:24:06.654 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2024-04-18T22:24:06.654 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-04-18T22:24:06.654 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.1 doing powercycle of ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:24:06.655 INFO:teuthology.orchestra.console.smithi096:Power on 2024-04-18T22:24:06.655 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T22:24:06.665 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:06.665 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:06.677 DEBUG:teuthology.orchestra.console.smithi096:power on output: Chassis Power Control: Up/On 2024-04-18T22:24:06.677 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T22:24:06.686 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:06.701 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2024-04-18T22:24:06.731 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:06.728+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:06.776 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:06.794 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:06.794 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:06.797 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:06.797 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:06.832 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:06.830+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.889 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.772116366000006, 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:06.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.772047767999993, 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.895 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:06.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771951922, 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:06.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771895040000004, 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.906 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.911 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771835367999998, 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:06.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:06.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:06.916 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771777405999998, 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:06.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:06.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:06.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:06.922 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771557713999997, 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:06.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:06.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:06.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771500785000001, 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:24:06.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:24:06.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:24:06.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:24:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771404899000004, 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:24:06.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:24:06.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:24:06.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:24:06.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771342571000005, 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:24:06.938 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.943 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771270092999998, 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:06.948 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771117625000002, 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:06.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.771038902000001, 2024-04-18T22:24:06.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:06.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:24:06.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.958 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.769518046000002, 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:06.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:06.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 94.769419694000007, 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:06.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.087697740999999, 2024-04-18T22:24:06.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:06.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.087043528000002, 2024-04-18T22:24:06.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:06.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.086936833000003, 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:06.979 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.085070924, 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:06.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:06.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 61.084405601, 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:06.984 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:06.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:06.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:06.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:06.988 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:06.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:06.990 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:06.984+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:07.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:24:07.091 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:07.088+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146723 5.c 5:3ca5ca4e:::benchmark_data_smithi032_22800_object18609:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.137185+0000", 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0074431059999999997, 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074676849999999999, 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146723 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137185+0000", 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137185+0000", 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137185+0000", 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6499999999999999e-07 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137188+0000", 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.785e-06 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137195+0000", 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5849999999999996e-06 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137200+0000", 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2839999999999997e-06 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137247+0000", 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7107000000000001e-05 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137366+0000", 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118705 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.137476+0000", 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109617 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142247+0000", 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047711129999999996 2024-04-18T22:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:146724 5.5 e51/49 5:a6d9ffe0:::benchmark_data_smithi032_22800_object18610:head v 51'9042, mlcod=51'9039)", 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.139166+0000", 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0054619559999999996, 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055670579999999997, 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139166+0000", 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139166+0000", 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139167+0000", 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7999999999999996e-07 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139172+0000", 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9389999999999996e-06 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139179+0000", 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4950000000000002e-06 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139184+0000", 2024-04-18T22:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7890000000000002e-06 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139231+0000", 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7531000000000003e-05 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139260+0000", 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9141000000000001e-05 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146727 5.a 5:56972f38:::benchmark_data_smithi032_22800_object18613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.139327+0000", 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0053014869999999997, 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054648140000000001, 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146727 2024-04-18T22:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139327+0000", 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139327+0000", 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139327+0000", 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9400000000000004e-07 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139332+0000", 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7600000000000002e-06 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139342+0000", 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0242e-05 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139358+0000", 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5668999999999999e-05 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139408+0000", 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9496999999999997e-05 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139483+0000", 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5108e-05 2024-04-18T22:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146728 5.a 5:531de7db:::benchmark_data_smithi032_22800_object18614:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.139520+0000", 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0051082599999999999, 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053337200000000001, 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146728 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139520+0000", 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139520+0000", 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139520+0000", 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5199999999999996e-07 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139525+0000", 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2350000000000001e-06 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139534+0000", 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4960000000000002e-06 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.139539+0000", 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5950000000000001e-06 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.140935+0000", 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013966619999999999 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141083+0000", 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000147143 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146730 5.a 5:537fbfb6:::benchmark_data_smithi032_22800_object18616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.141802+0000", 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0028263569999999998, 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031154809999999998, 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146730 2024-04-18T22:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141802+0000", 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141802+0000", 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141802+0000", 2024-04-18T22:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.37e-07 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141806+0000", 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9589999999999997e-06 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141813+0000", 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141820+0000", 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8789999999999997e-06 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142527+0000", 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00070677499999999998 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142597+0000", 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9706e-05 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146731 5.2 5:4e5328d1:::benchmark_data_smithi032_22800_object18617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.141845+0000", 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002782844, 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031327070000000002, 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146731 2024-04-18T22:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141845+0000", 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141845+0000", 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141846+0000", 2024-04-18T22:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7100000000000002e-07 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141849+0000", 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9799999999999998e-06 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141858+0000", 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9299999999999992e-06 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141863+0000", 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3079999999999998e-06 2024-04-18T22:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142519+0000", 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00065602399999999995 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142639+0000", 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000119695 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142746+0000", 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010688000000000001 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146732 5.b 5:db831a16:::benchmark_data_smithi032_22800_object18618:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.141871+0000", 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002756784, 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031734929999999999, 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146732 2024-04-18T22:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141871+0000", 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141871+0000", 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141872+0000", 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0600000000000001e-07 2024-04-18T22:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141874+0000", 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1799999999999999e-06 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141883+0000", 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7050000000000005e-06 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141888+0000", 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5010000000000004e-06 2024-04-18T22:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141926+0000", 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8321000000000001e-05 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142014+0000", 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7923000000000005e-05 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146733 5.b 5:d1509344:::benchmark_data_smithi032_22800_object18619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.141902+0000", 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002725594, 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032048549999999999, 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146733 2024-04-18T22:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141902+0000", 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141902+0000", 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141903+0000", 2024-04-18T22:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7399999999999998e-07 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141905+0000", 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.215e-06 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141912+0000", 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7000000000000002e-06 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.141917+0000", 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7809999999999996e-06 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.143909+0000", 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019924420000000001 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.143993+0000", 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3792e-05 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146734 5.7 5:eb048566:::benchmark_data_smithi032_22800_object18620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.142387+0000", 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022414079999999999, 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027819379999999999, 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146734 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142387+0000", 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142387+0000", 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142387+0000", 2024-04-18T22:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8400000000000003e-07 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142392+0000", 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4689999999999999e-06 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142402+0000", 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0198e-05 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142410+0000", 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142454+0000", 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3900999999999999e-05 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142562+0000", 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000108181 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142661+0000", 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9426000000000007e-05 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146735 5.b 5.3d4ae52b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.142423+0000", 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022047600000000001, 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028093369999999999, 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146735 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142423+0000", 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142423+0000", 2024-04-18T22:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142424+0000", 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0299999999999999e-07 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142426+0000", 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4849999999999999e-06 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142433+0000", 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3339999999999998e-06 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142438+0000", 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2700000000000004e-06 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146736 5.c 5:34feb239:::benchmark_data_smithi032_22800_object18622:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.142456+0000", 2024-04-18T22:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021723900000000002, 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028274229999999999, 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146736 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142456+0000", 2024-04-18T22:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142456+0000", 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142456+0000", 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3799999999999997e-07 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142459+0000", 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7010000000000001e-06 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142465+0000", 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3389999999999998e-06 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.142470+0000", 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7550000000000002e-06 2024-04-18T22:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144138+0000", 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001668159 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144261+0000", 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122593 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144374+0000", 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113615 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146737 5.2 5:4907567f:::benchmark_data_smithi032_22800_object18623:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.144118+0000", 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00050995399999999996, 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012308250000000001, 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146737 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144118+0000", 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144118+0000", 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144119+0000", 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144127+0000", 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9780000000000005e-06 2024-04-18T22:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144151+0000", 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3747000000000002e-05 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144158+0000", 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9190000000000002e-06 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144596+0000", 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00043807900000000002 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144668+0000", 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2539999999999993e-05 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144727+0000", 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9182999999999999e-05 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:146739 5.e 5:74c58236:::benchmark_data_smithi032_22800_object18625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:07.144173+0000", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00045491100000000001, 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001242258, 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 146739 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144173+0000", 2024-04-18T22:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144173+0000", 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144174+0000", 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7899999999999998e-07 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144177+0000", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.96e-06 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144183+0000", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2589999999999997e-06 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144188+0000", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999997e-06 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144299+0000", 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011072499999999999 2024-04-18T22:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:07.144367+0000", 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7648e-05 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 13 2024-04-18T22:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:07.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:07.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:07.198+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:07.300 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:07.298+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:07.396 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:07.396 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:07.396 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:07.396 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:07.396 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:07.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:07.398+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:07.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:07.503 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:07.500+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:07.567 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:07.567 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:07.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:07.567 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.766626852000002, 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:07.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:07.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:07.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.764970472999998, 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.764845756, 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.763066098000003, 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.585 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:07.590 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.762583427000003, 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.762344352, 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:07.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.762223581000001, 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:07.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.759334168999999, 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.607 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:07.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:07.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:07.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.759212349999999, 2024-04-18T22:24:07.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:07.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:07.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.759101901000001, 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:07.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:07.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:07.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:07.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.759032433000002, 2024-04-18T22:24:07.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:07.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:07.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:07.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 61.758975292000002, 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.629 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:07.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:07.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:07.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.634 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5297997810000004, 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5251567819999998, 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:07.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:07.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5160590450000004, 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:07.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:07.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.50703912, 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:07.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:07.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.4910277519999999, 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:07.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.4882243319999997, 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.665 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:07.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:07.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:07.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137384 5.0 5:0d4b5451:::benchmark_data_smithi032_22800_object9270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.4428477639999997, 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.054059477000000002, 2024-04-18T22:24:07.671 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137384 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122528+0000", 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1629999999999999e-06 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122533+0000", 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5570000000000003e-06 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122554+0000", 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1222000000000001e-05 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122571+0000", 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7005000000000001e-05 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156535+0000", 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.033963356 2024-04-18T22:24:07.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156833+0000", 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00029849800000000001 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.157060+0000", 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022695300000000001 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175844+0000", 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018784072999999998 2024-04-18T22:24:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176295+0000", 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00045056399999999998 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176394+0000", 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9667000000000006e-05 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176586+0000", 2024-04-18T22:24:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019141899999999999 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137394 5.0 5:082322e3:::benchmark_data_smithi032_22800_object9280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.4324980270000003, 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.050257665, 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:07.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137394 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132878+0000", 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6169999999999999e-06 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132885+0000", 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.126e-06 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132905+0000", 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0369000000000001e-05 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132921+0000", 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172761+0000", 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039840484000000002 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172994+0000", 2024-04-18T22:24:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023306499999999999 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.173221+0000", 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000226949 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.179433+0000", 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062120439999999999 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182890+0000", 2024-04-18T22:24:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034568089999999999 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182980+0000", 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9844000000000002e-05 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.183134+0000", 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015365800000000001 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:07.683 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:07.683 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:07.684 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:07.685 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:07.680+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:07.785 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:07.780+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148397 5.f 5:f23d62b3:::benchmark_data_smithi032_22800_object20283:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.834248+0000", 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0049755040000000004, 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050009140000000004, 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.840 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148397 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834248+0000", 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834248+0000", 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834249+0000", 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0240000000000001e-06 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834254+0000", 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.101e-06 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834273+0000", 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9139000000000001e-05 2024-04-18T22:24:07.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834280+0000", 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2910000000000002e-06 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.835067+0000", 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00078737299999999996 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.835146+0000", 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8374999999999994e-05 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.843 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148398 5.1 5:80298678:::benchmark_data_smithi032_22800_object20284:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.834297+0000", 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0049262070000000002, 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0050249530000000004, 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148398 2024-04-18T22:24:07.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834297+0000", 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834297+0000", 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834297+0000", 2024-04-18T22:24:07.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7899999999999999e-07 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834300+0000", 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.728e-06 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834308+0000", 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2290000000000003e-06 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834313+0000", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0690000000000001e-06 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834364+0000", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1063999999999998e-05 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834514+0000", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014956999999999999 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834636+0000", 2024-04-18T22:24:07.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000121592 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836697+0000", 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020612640000000002 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:148396 5.3 e51/19 5:c151486b:::benchmark_data_smithi032_22800_object20282:head v 51'9489, mlcod=51'9487)", 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.834471+0000", 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.004751618, 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049299160000000003, 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.848 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834471+0000", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834471+0000", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834472+0000", 2024-04-18T22:24:07.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7399999999999998e-07 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834479+0000", 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1640000000000004e-06 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834495+0000", 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6209e-05 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834501+0000", 2024-04-18T22:24:07.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7370000000000003e-06 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834544+0000", 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3381999999999999e-05 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834564+0000", 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9494e-05 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.851 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:148399 5.3 e51/19 5:c72002c4:::benchmark_data_smithi032_22800_object20285:head v 51'9490, mlcod=51'9487)", 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.836386+0000", 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002837015, 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0030748310000000001, 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836386+0000", 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836386+0000", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836387+0000", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4000000000000001e-07 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836391+0000", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2830000000000003e-06 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836399+0000", 2024-04-18T22:24:07.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4130000000000006e-06 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836404+0000", 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0130000000000001e-06 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836452+0000", 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7706e-05 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836466+0000", 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4127999999999999e-05 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148401 5.5 5:a63e73b6:::benchmark_data_smithi032_22800_object20287:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.836843+0000", 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002380058, 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002678606, 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.855 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148401 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836843+0000", 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836843+0000", 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836844+0000", 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4200000000000005e-07 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836848+0000", 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8650000000000003e-06 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836858+0000", 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0727000000000001e-05 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836866+0000", 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2980000000000003e-06 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.836917+0000", 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0970000000000002e-05 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837003+0000", 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6700999999999995e-05 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837072+0000", 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8250000000000006e-05 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148403 5.5 5:a9c2dfe1:::benchmark_data_smithi032_22800_object20289:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.837830+0000", 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0013935219999999999, 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017616050000000001, 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.861 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148403 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837830+0000", 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837830+0000", 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837830+0000", 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6499999999999999e-07 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837833+0000", 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0489999999999999e-06 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837844+0000", 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0523000000000001e-05 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837849+0000", 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0089999999999998e-06 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838785+0000", 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00093573899999999999 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838978+0000", 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000192976 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839116+0000", 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000138477 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148404 5.5 5.f58fe465 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.837858+0000", 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001364774, 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001798596, 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.865 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148404 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837858+0000", 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837858+0000", 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837859+0000", 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1600000000000002e-07 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837860+0000", 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.584e-06 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837865+0000", 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9269999999999996e-06 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837868+0000", 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9059999999999998e-06 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148407 5.f 5:f627fe9a:::benchmark_data_smithi032_22800_object20293:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.837973+0000", 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001250447, 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017422259999999999, 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.868 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148407 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837973+0000", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837973+0000", 2024-04-18T22:24:07.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837973+0000", 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9200000000000001e-07 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837976+0000", 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6070000000000002e-06 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837982+0000", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0630000000000002e-06 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837985+0000", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.219e-06 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838030+0000", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4790999999999999e-05 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838162+0000", 2024-04-18T22:24:07.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013222599999999999 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148408 5.9 5:9e206b77:::benchmark_data_smithi032_22800_object20294:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.837992+0000", 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001231212, 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017858749999999999, 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.872 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148408 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837992+0000", 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837992+0000", 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837992+0000", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2700000000000001e-07 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837994+0000", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9190000000000002e-06 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.837997+0000", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3239999999999999e-06 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838000+0000", 2024-04-18T22:24:07.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6520000000000002e-06 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838055+0000", 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5003000000000002e-05 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838182+0000", 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012740200000000001 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.875 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148409 5.f 5.53467faf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.838006+0000", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00121688, 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018305800000000001, 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148409 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838006+0000", 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838006+0000", 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838006+0000", 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.23e-07 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838008+0000", 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3239999999999999e-06 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838011+0000", 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.461e-06 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838014+0000", 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8229999999999999e-06 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:148406 5.c e51/19 5:348f1016:::benchmark_data_smithi032_22800_object20292:head v 51'9346, mlcod=51'9344)", 2024-04-18T22:24:07.878 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.838115+0000", 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001107678, 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017720629999999999, 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838115+0000", 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838115+0000", 2024-04-18T22:24:07.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838116+0000", 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9700000000000002e-07 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838119+0000", 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.3400000000000002e-06 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838124+0000", 2024-04-18T22:24:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4630000000000003e-06 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838127+0000", 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1350000000000001e-06 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838164+0000", 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7409e-05 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838177+0000", 2024-04-18T22:24:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2665e-05 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:148405 5.6 e51/19 5:6fc3f2a4:::benchmark_data_smithi032_22800_object20291:head v 51'9196, mlcod=51'9194)", 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.838253+0000", 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000969811, 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0016880280000000001, 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838253+0000", 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838253+0000", 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838254+0000", 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5599999999999995e-07 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838258+0000", 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4819999999999996e-06 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838266+0000", 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1580000000000005e-06 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838272+0000", 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5210000000000002e-06 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838327+0000", 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4789e-05 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838346+0000", 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9721e-05 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop_reply(client.4586.0:148401 5.5 e51/49)", 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.838878+0000", 2024-04-18T22:24:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00034466500000000002, 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011137320000000001, 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838878+0000", 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838878+0000", 2024-04-18T22:24:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838879+0000", 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1900000000000002e-07 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838883+0000", 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8800000000000001e-06 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838891+0000", 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3920000000000003e-06 2024-04-18T22:24:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.838897+0000", 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4960000000000004e-06 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148410 5.1 5:892429ab:::benchmark_data_smithi032_22800_object20296:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.839096+0000", 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00012715400000000001, 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00094312200000000001, 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148410 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839096+0000", 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839096+0000", 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839097+0000", 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9999999999999997e-07 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839100+0000", 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5810000000000001e-06 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839111+0000", 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0643e-05 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839117+0000", 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8459999999999996e-06 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839163+0000", 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6650999999999998e-05 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839281+0000", 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011746199999999999 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.839388+0000", 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010677499999999999 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148394 5.5 5:ac01d20e:::benchmark_data_smithi032_22800_object20280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.892 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.832559+0000", 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0066637650000000003, 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0075502570000000003, 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148394 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832559+0000", 2024-04-18T22:24:07.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832559+0000", 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832560+0000", 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.51e-07 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832563+0000", 2024-04-18T22:24:07.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8439999999999998e-06 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832570+0000", 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.559e-06 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832574+0000", 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9999999999999998e-06 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832619+0000", 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4944000000000002e-05 2024-04-18T22:24:07.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832755+0000", 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013557700000000001 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.832883+0000", 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012869300000000001 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.834578+0000", 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001694269 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.896 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:148395 5.f 5:f4e81be7:::benchmark_data_smithi032_22800_object20281:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:07.833023+0000", 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0061996009999999999, 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0071702119999999996, 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:07.897 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 148395 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833023+0000", 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833023+0000", 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:07.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833024+0000", 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5700000000000001e-07 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833029+0000", 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8670000000000002e-06 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833044+0000", 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4395e-05 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:07.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833051+0000", 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1169999999999998e-06 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833119+0000", 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8312999999999995e-05 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:07.833245+0000", 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012630599999999999 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:07.901 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:07.901 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 16 2024-04-18T22:24:07.901 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:07.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:07.903 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:07.896+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:08.002 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:24:08.003 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:08.000+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:08.054 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:08.054 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:08.054 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:08.054 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:08.054 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:08.074 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:08.103 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:08.100+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:08.204 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:08.200+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.116168565999999, 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.232 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:08.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.237 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.116099968, 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.116004122000007, 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:08.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115947239999997, 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.248 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:08.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:08.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115887568000005, 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:08.258 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115829606000005, 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115609914000004, 2024-04-18T22:24:08.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.270 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115552984999994, 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:24:08.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115457098999997, 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:08.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:24:08.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115394770999998, 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.281 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:24:08.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:24:08.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:24:08.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:24:08.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115322293000006, 2024-04-18T22:24:08.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:08.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:08.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.291 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115169824999995, 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:08.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:08.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:08.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.115091101999994, 2024-04-18T22:24:08.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:24:08.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:24:08.302 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.113570245999995, 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:08.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 96.113471894, 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.308 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 62.431749941, 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.312 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:08.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:08.316 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 62.431095728000003, 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:08.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:08.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 62.430989033000003, 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:08.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:08.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 62.429123124, 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.325 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:08.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.329 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 62.428457801, 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.333 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.334 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.334 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:08.334 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:08.334 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:08.335 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:08.334+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:08.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:08.436 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:08.432+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:08.490 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150010 5.c 5:34acc4eb:::benchmark_data_smithi032_22800_object21896:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.480547+0000", 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0092016949999999993, 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0092241140000000003, 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150010 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480547+0000", 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480547+0000", 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480547+0000", 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480550+0000", 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7800000000000001e-06 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480557+0000", 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6490000000000002e-06 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480562+0000", 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3329999999999996e-06 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480606+0000", 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3417000000000001e-05 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480736+0000", 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130891 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480853+0000", 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116753 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487520+0000", 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066665079999999998 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489773+0000", 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00225355 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150011 5.a 5:518c4c5d:::benchmark_data_smithi032_22800_object21897:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.480574+0000", 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0091746280000000006, 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0092958699999999995, 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150011 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.496 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480574+0000", 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480574+0000", 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480574+0000", 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.41e-07 2024-04-18T22:24:08.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480576+0000", 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1579999999999999e-06 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480582+0000", 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8499999999999999e-06 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.480587+0000", 2024-04-18T22:24:08.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6229999999999996e-06 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.481525+0000", 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00093791700000000003 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.481653+0000", 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012837800000000001 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150015 5.7 5:eb995e13:::benchmark_data_smithi032_22800_object21901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.482329+0000", 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0074195299999999997, 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0076135329999999996, 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150015 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482329+0000", 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482329+0000", 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482329+0000", 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4999999999999998e-07 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482332+0000", 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5550000000000001e-06 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482344+0000", 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1579e-05 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482349+0000", 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.293e-06 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482406+0000", 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7086000000000002e-05 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482533+0000", 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012718699999999999 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.482632+0000", 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8567999999999995e-05 2024-04-18T22:24:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486753+0000", 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041215310000000003 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150018 5.7 5:e741f372:::benchmark_data_smithi032_22800_object21904:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.483488+0000", 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.006260425, 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0065373300000000001, 2024-04-18T22:24:08.504 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150018 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.483488+0000", 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.483488+0000", 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.483489+0000", 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9999999999999996e-07 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.483494+0000", 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6670000000000004e-06 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.483503+0000", 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.448e-06 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.483509+0000", 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2700000000000001e-06 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484522+0000", 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001012502 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484649+0000", 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012744699999999999 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484765+0000", 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115654 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487510+0000", 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027454559999999999 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150021 5.c 5:3d8c7863:::benchmark_data_smithi032_22800_object21907:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.484365+0000", 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0053832560000000003, 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057414450000000004, 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150021 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484365+0000", 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484365+0000", 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484366+0000", 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0399999999999996e-07 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484368+0000", 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6790000000000001e-06 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484376+0000", 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9240000000000007e-06 2024-04-18T22:24:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484382+0000", 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1319999999999999e-06 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484672+0000", 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00028947599999999999 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.484788+0000", 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011624399999999999 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485098+0000", 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031016000000000003 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150023 5.a 5:5db06a59:::benchmark_data_smithi032_22800_object21909:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.485281+0000", 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0044671909999999997, 2024-04-18T22:24:08.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048950490000000003, 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150023 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485281+0000", 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485281+0000", 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485282+0000", 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1999999999999999e-07 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485286+0000", 2024-04-18T22:24:08.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0969999999999999e-06 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485295+0000", 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3179999999999996e-06 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.485302+0000", 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2650000000000002e-06 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486536+0000", 2024-04-18T22:24:08.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012338729999999999 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486671+0000", 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013558099999999999 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150024 5.a 5:5899289f:::benchmark_data_smithi032_22800_object21910:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.486664+0000", 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030841739999999999, 2024-04-18T22:24:08.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003575286, 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150024 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486664+0000", 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486664+0000", 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486665+0000", 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999996e-07 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486670+0000", 2024-04-18T22:24:08.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4429999999999996e-06 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486678+0000", 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.85e-06 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.486686+0000", 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2359999999999998e-06 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489877+0000", 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031916710000000001 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489982+0000", 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010477800000000001 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150025 5.6 5:6d6d2603:::benchmark_data_smithi032_22800_object21911:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.487067+0000", 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026818110000000001, 2024-04-18T22:24:08.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003233816, 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150025 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487067+0000", 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487067+0000", 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487067+0000", 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1500000000000004e-07 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487071+0000", 2024-04-18T22:24:08.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5109999999999999e-06 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487078+0000", 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1049999999999997e-06 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487083+0000", 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.78e-06 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487843+0000", 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00076011099999999995 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487943+0000", 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9968000000000002e-05 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488021+0000", 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8125000000000002e-05 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150026 5.4 5:2ca51083:::benchmark_data_smithi032_22800_object21912:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.487818+0000", 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0019300319999999999, 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025482209999999998, 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150026 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487818+0000", 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487818+0000", 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487819+0000", 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4400000000000001e-07 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487822+0000", 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.689e-06 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487827+0000", 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3499999999999996e-06 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487831+0000", 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9369999999999997e-06 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487873+0000", 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1850000000000001e-05 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487980+0000", 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010770899999999999 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150027 5.4 5:238e0ef0:::benchmark_data_smithi032_22800_object21913:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.529 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.487841+0000", 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0019077339999999999, 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025861399999999998, 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150027 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487841+0000", 2024-04-18T22:24:08.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487841+0000", 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487841+0000", 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3099999999999998e-07 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487844+0000", 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7539999999999998e-06 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487850+0000", 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1589999999999998e-06 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487856+0000", 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3090000000000001e-06 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489745+0000", 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018889029999999999 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489875+0000", 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012951400000000001 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150028 5.3 5:c248756a:::benchmark_data_smithi032_22800_object21914:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.487874+0000", 2024-04-18T22:24:08.533 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0018745330000000001, 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026135889999999999, 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150028 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487874+0000", 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487874+0000", 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487875+0000", 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2699999999999999e-07 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487877+0000", 2024-04-18T22:24:08.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.756e-06 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487887+0000", 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4900000000000006e-06 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487892+0000", 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0130000000000001e-06 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487940+0000", 2024-04-18T22:24:08.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8152000000000002e-05 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488064+0000", 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012430300000000001 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488176+0000", 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112038 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150029 5.6 5:69fe6c6b:::benchmark_data_smithi032_22800_object21915:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.487904+0000", 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001844432, 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026509279999999999, 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150029 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487904+0000", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487904+0000", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487905+0000", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4499999999999997e-07 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487907+0000", 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4509999999999999e-06 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487915+0000", 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.763e-06 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.487919+0000", 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.566e-06 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489656+0000", 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001736776 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489787+0000", 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000131199 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489907+0000", 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012013 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150030 5.b 5:d8437c96:::benchmark_data_smithi032_22800_object21916:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.488288+0000", 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014600139999999999, 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023334480000000001, 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150030 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488288+0000", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488288+0000", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488289+0000", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3699999999999999e-07 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488294+0000", 2024-04-18T22:24:08.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5109999999999997e-06 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488304+0000", 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7669999999999998e-06 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488311+0000", 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0859999999999996e-06 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488424+0000", 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113402 2024-04-18T22:24:08.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488554+0000", 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012948699999999999 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150031 5.8 5:1de74999:::benchmark_data_smithi032_22800_object21917:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.488329+0000", 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014195659999999999, 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002357456, 2024-04-18T22:24:08.545 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150031 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488329+0000", 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488329+0000", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488329+0000", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6000000000000004e-07 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488332+0000", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9000000000000002e-06 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488341+0000", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7290000000000006e-06 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488347+0000", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6879999999999996e-06 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489752+0000", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014056019999999999 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.489861+0000", 2024-04-18T22:24:08.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010818 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150032 5.6 5.27af26c6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.488359+0000", 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001389157, 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023855669999999999, 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.549 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150032 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488359+0000", 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488359+0000", 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488360+0000", 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5600000000000001e-07 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488362+0000", 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5519999999999999e-06 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488370+0000", 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1189999999999997e-06 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.488376+0000", 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.164e-06 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:150033 5.0 5.6cc44c30 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.490236+0000", 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9995122, 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055959800000000002, 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 150033 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490236+0000", 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490236+0000", 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490237+0000", 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.167e-06 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490241+0000", 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0949999999999998e-06 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490253+0000", 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1497e-05 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490260+0000", 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7510000000000002e-06 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(5.0 52/49 [PushReplyOp(5:01ce2202:::benchmark_data_smithi032_22800_object48425:head)])", 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:08.490614+0000", 2024-04-18T22:24:08.555 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9991345, 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023094500000000001, 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490614+0000", 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490614+0000", 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490614+0000", 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1500000000000001e-07 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490617+0000", 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.128e-06 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490626+0000", 2024-04-18T22:24:08.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2900000000000002e-06 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:08.490631+0000", 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5659999999999997e-06 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 17 2024-04-18T22:24:08.558 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:08.559 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:08.558+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:08.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:08.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:08.658+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:08.751 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:08.751 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:08.751 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:08.751 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:08.751 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:08.761 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:08.758+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:08.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:08.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:08.861 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:08.856+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:08.926 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.126410024999998, 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:08.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.124753646000002, 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:08.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:08.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:08.938 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.124628928999996, 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:08.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:08.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.122849271, 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.944 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:08.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:08.949 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.122366599999999, 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:08.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:08.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:08.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.122127525000003, 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.955 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:08.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:08.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.960 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.122006753999997, 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:08.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:08.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:08.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.119117342000003, 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.118995523000002, 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.118885073999998, 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:08.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.118815605999998, 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:08.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 63.118758464999999, 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:08.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:08.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.8895829539999998, 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:08.995 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:08.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:08.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:08.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:08.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.8849399550000001, 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:09.001 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:09.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.8758422179999998, 2024-04-18T22:24:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:09.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.8668222930000002, 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.014 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:09.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:09.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.8508109250000002, 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:09.020 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.026 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.848007505, 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.027 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:09.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:09.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137384 5.0 5:0d4b5451:::benchmark_data_smithi032_22800_object9270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.802630937, 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.054059477000000002, 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137384 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122528+0000", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1629999999999999e-06 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122533+0000", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5570000000000003e-06 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122554+0000", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1222000000000001e-05 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122571+0000", 2024-04-18T22:24:09.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7005000000000001e-05 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156535+0000", 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.033963356 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156833+0000", 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00029849800000000001 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.157060+0000", 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022695300000000001 2024-04-18T22:24:09.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175844+0000", 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018784072999999998 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176295+0000", 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00045056399999999998 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176394+0000", 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9667000000000006e-05 2024-04-18T22:24:09.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176586+0000", 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019141899999999999 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137394 5.0 5:082322e3:::benchmark_data_smithi032_22800_object9280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 5.7922811999999997, 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.050257665, 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:09.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137394 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:09.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132878+0000", 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6169999999999999e-06 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132885+0000", 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.126e-06 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:09.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132905+0000", 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0369000000000001e-05 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132921+0000", 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172761+0000", 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039840484000000002 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:09.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172994+0000", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023306499999999999 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.173221+0000", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000226949 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.179433+0000", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062120439999999999 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182890+0000", 2024-04-18T22:24:09.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034568089999999999 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182980+0000", 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9844000000000002e-05 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.183134+0000", 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015365800000000001 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:09.043 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:09.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:09.046 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:09.046 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:09.047 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.040+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:09.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:09.048 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:09.048 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:09.146 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.140+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:09.196 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:09.196 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:09.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:09.247 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.244+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:09.349 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.344+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:09.370 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:09.370 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:09.390 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:09.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~35s 2024-04-18T22:24:09.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:09.446+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:09.541 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:09.542 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:09.551 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.548+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:09.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:09.652 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.648+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:09.713 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:09.713 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:09.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:09.752 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:09.750+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:09.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:09.850+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:09.920 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:09.920 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:09.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:09.987 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:09.984+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:10.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:24:10.055 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:10.054+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:10.093 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:10.093 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:10.113 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:10.156 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:10.152+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:10.256 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:10.254+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:10.263 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:10.263 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:10.284 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:10.358 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:10.352+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:10.458 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:10.456+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:10.509 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:10.561 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:10.556+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:10.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:10.658+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:10.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:10.701 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T22:24:10.760 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:10.756+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:10.812 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-04-18T22:24:10.826 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:10.826 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:10.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:10.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:10.858+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:10.913 INFO:teuthology.orchestra.console.smithi096:Power on completed 2024-04-18T22:24:10.962 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:10.960+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:11.014 DEBUG:teuthology.orchestra.console.smithi096:Waiting for login prompt 2024-04-18T22:24:11.014 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi096 2024-04-18T22:24:11.023 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:24:11.038 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:11.039 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:11.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:11.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:11.099 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:11.096+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:11.164 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:11.162+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:11.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:11.225 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:11.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:11.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:11.229 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:11.252 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.451739947999997, 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:11.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:11.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.450083569, 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:11.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:11.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.264 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.449958851999995, 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:11.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:11.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:11.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.448179194000005, 2024-04-18T22:24:11.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.447696523000005, 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:11.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.447457447999994, 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:11.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:11.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.447336676999996, 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:11.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:11.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.444447264999994, 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:11.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:11.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:11.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.444325445999993, 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.299 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:11.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:11.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.444214997000003, 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:11.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:11.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:11.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:11.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.444145528999996, 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:11.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:11.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 65.444088387999997, 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:11.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:11.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.2149128769999997, 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.327 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.2102698780000001, 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:11.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:11.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.2011721410000007, 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:11.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.337 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:11.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:11.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.1921522160000002, 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.343 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.1761408479999993, 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:11.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:11.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:11.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:11.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.173337428, 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:11.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.360 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.361 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137384 5.0 5:0d4b5451:::benchmark_data_smithi032_22800_object9270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.12796086, 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.054059477000000002, 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137384 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122528+0000", 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1629999999999999e-06 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122533+0000", 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5570000000000003e-06 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122554+0000", 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1222000000000001e-05 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122571+0000", 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7005000000000001e-05 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156535+0000", 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.033963356 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156833+0000", 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00029849800000000001 2024-04-18T22:24:11.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.157060+0000", 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022695300000000001 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175844+0000", 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018784072999999998 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176295+0000", 2024-04-18T22:24:11.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00045056399999999998 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176394+0000", 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9667000000000006e-05 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176586+0000", 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019141899999999999 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137394 5.0 5:082322e3:::benchmark_data_smithi032_22800_object9280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.1176111229999997, 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.050257665, 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137394 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.368 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132878+0000", 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6169999999999999e-06 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132885+0000", 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.126e-06 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132905+0000", 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0369000000000001e-05 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132921+0000", 2024-04-18T22:24:11.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172761+0000", 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039840484000000002 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172994+0000", 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023306499999999999 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.173221+0000", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000226949 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.179433+0000", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062120439999999999 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182890+0000", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034568089999999999 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:11.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182980+0000", 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9844000000000002e-05 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.183134+0000", 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015365800000000001 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:11.373 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:11.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:11.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:11.370+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:11.475 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:11.472+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop_reply(client.4586.0:157204 5.5 e53/49 ondisk, result = 0)", 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.529364+0000", 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001300701, 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001310767, 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.531 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529364+0000", 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529364+0000", 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529374+0000", 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0718e-05 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.532 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529378+0000", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.613e-06 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529390+0000", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2102e-05 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529398+0000", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1510000000000004e-06 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.533 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.530652+0000", 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012540120000000001 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.530659+0000", 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7100000000000001e-06 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.530674+0000", 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5461e-05 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157210 5.7 e53/23 5:e59c3f7f:::benchmark_data_smithi032_22800_object29096:head v 53'9905, mlcod=53'9901)", 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.529431+0000", 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001233289, 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012916550000000001, 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529431+0000", 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529431+0000", 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529432+0000", 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0600000000000001e-07 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529436+0000", 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5650000000000003e-06 2024-04-18T22:24:11.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529446+0000", 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5310000000000007e-06 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529454+0000", 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0499999999999992e-06 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529841+0000", 2024-04-18T22:24:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038700600000000001 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529861+0000", 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0724999999999999e-05 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157208 5.3 e53/19 5:cb8c18a4:::benchmark_data_smithi032_22800_object29094:head v 53'10062, mlcod=53'10059)", 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.529818+0000", 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00084593199999999998, 2024-04-18T22:24:11.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00093767000000000004, 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529818+0000", 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529818+0000", 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529819+0000", 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0999999999999999e-07 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529823+0000", 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1799999999999998e-06 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529833+0000", 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9799999999999993e-06 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529839+0000", 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3199999999999996e-06 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529880+0000", 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0930000000000003e-05 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529898+0000", 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7538e-05 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:157212 5.5 5:a14d0beb:::benchmark_data_smithi032_22800_object29098:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.529884+0000", 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00078027299999999995, 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00090500499999999996, 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-04-18T22:24:11.542 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 157212 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529884+0000", 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529884+0000", 2024-04-18T22:24:11.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529885+0000", 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8000000000000004e-07 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529890+0000", 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5040000000000004e-06 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529900+0000", 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0172e-05 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529907+0000", 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.199e-06 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.530690+0000", 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00078350399999999997 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.545 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:157194 5.1 5:869e8db6:::benchmark_data_smithi032_22800_object29080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.523527+0000", 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.007137722, 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072993670000000002, 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:11.546 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 157194 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523527+0000", 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523527+0000", 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.547 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523527+0000", 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1699999999999997e-07 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523531+0000", 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8449999999999996e-06 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523542+0000", 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0958000000000001e-05 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523547+0000", 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2889999999999997e-06 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.525534+0000", 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001986706 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.525667+0000", 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013264599999999999 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.525841+0000", 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000173995 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529158+0000", 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033174060000000002 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:157199 5.d 5:bb25d3c0:::benchmark_data_smithi032_22800_object29085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.523701+0000", 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0069636979999999999, 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0071800040000000002, 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 157199 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523701+0000", 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523701+0000", 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523702+0000", 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1510000000000001e-06 2024-04-18T22:24:11.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523706+0000", 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9020000000000001e-06 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523714+0000", 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4640000000000006e-06 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523722+0000", 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7440000000000007e-06 2024-04-18T22:24:11.553 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523757+0000", 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5036000000000001e-05 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523833+0000", 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6045999999999997e-05 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.554 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157198 5.7 e53/23 5:e5b7c367:::benchmark_data_smithi032_22800_object29084:head v 53'9903, mlcod=53'9901)", 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.523986+0000", 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0066781940000000001, 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0069302909999999999, 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523986+0000", 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.555 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523986+0000", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523987+0000", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6600000000000002e-07 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523990+0000", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2100000000000002e-06 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.523996+0000", 2024-04-18T22:24:11.556 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4110000000000001e-06 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.524001+0000", 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4939999999999997e-06 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.524043+0000", 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2227000000000001e-05 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.524062+0000", 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9134999999999999e-05 2024-04-18T22:24:11.557 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157201 5.7 e53/23 5:e0ace8cb:::benchmark_data_smithi032_22800_object29087:head v 53'9904, mlcod=53'9901)", 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.526238+0000", 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0044267359999999997, 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.004726264, 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.558 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526238+0000", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526238+0000", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526238+0000", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8799999999999998e-07 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.559 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526242+0000", 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3959999999999999e-06 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526263+0000", 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0364999999999999e-05 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526269+0000", 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7590000000000003e-06 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.560 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526307+0000", 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8529999999999999e-05 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526324+0000", 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6631000000000001e-05 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157202 5.0 e53/49 5:04ad7827:::benchmark_data_smithi032_22800_object29088:head v 53'9773, mlcod=53'9771)", 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.526284+0000", 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0043804969999999997, 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0047400480000000002, 2024-04-18T22:24:11.561 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526284+0000", 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526284+0000", 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.562 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526284+0000", 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7899999999999999e-07 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526288+0000", 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9280000000000003e-06 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526295+0000", 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6050000000000003e-06 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.563 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526299+0000", 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6670000000000004e-06 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526659+0000", 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00035914600000000002 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526671+0000", 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2758000000000001e-05 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.564 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:157204 5.5 5:abc87902:::benchmark_data_smithi032_22800_object29090:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.526813+0000", 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0038518290000000002, 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042857909999999997, 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 157204 2024-04-18T22:24:11.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526813+0000", 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526813+0000", 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526814+0000", 2024-04-18T22:24:11.566 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.437e-06 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526819+0000", 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.023e-06 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526835+0000", 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5631999999999999e-05 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526844+0000", 2024-04-18T22:24:11.567 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3680000000000008e-06 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526900+0000", 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5529999999999999e-05 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527014+0000", 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000114361 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:11.568 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527079+0000", 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5170000000000001e-05 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.530666+0000", 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035870440000000002 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:157206 5.f 5:f18ccd3c:::benchmark_data_smithi032_22800_object29092:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.526866+0000", 2024-04-18T22:24:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0037979849999999998, 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042859300000000003, 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 157206 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526866+0000", 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.570 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526866+0000", 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526867+0000", 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2699999999999999e-07 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526871+0000", 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4570000000000001e-06 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.571 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526879+0000", 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0690000000000002e-06 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526884+0000", 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9179999999999999e-06 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.526930+0000", 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5021000000000001e-05 2024-04-18T22:24:11.572 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527060+0000", 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013041499999999999 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157205 5.6 e53/19 5:6fb40580:::benchmark_data_smithi032_22800_object29091:head v 53'9700, mlcod=53'9696)", 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.527189+0000", 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0034754270000000001, 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039985890000000003, 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.573 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527189+0000", 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527189+0000", 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.574 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527190+0000", 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5000000000000002e-07 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527194+0000", 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8749999999999999e-06 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527205+0000", 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0159999999999999e-05 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.575 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527211+0000", 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7750000000000004e-06 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527270+0000", 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8884000000000001e-05 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.527289+0000", 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.872e-05 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.576 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157207 5.3 e53/19 5:cd5ac88c:::benchmark_data_smithi032_22800_object29093:head v 53'10061, mlcod=53'10059)", 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.528112+0000", 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002552538, 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031065789999999999, 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528112+0000", 2024-04-18T22:24:11.577 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528112+0000", 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528112+0000", 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3600000000000004e-07 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.578 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528122+0000", 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6770000000000007e-06 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528132+0000", 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5599999999999999e-06 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528136+0000", 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7400000000000004e-06 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.579 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528179+0000", 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2481999999999997e-05 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528191+0000", 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1883e-05 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:157203 5.0 e53/49 5:05f08dab:::benchmark_data_smithi032_22800_object29089:head v 53'9774, mlcod=53'9771)", 2024-04-18T22:24:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.528477+0000", 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0021873610000000001, 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027720470000000001, 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528477+0000", 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528477+0000", 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.581 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528478+0000", 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-07 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528483+0000", 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2109999999999997e-06 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528493+0000", 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0519999999999999e-05 2024-04-18T22:24:11.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528501+0000", 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2869999999999999e-06 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528590+0000", 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8894999999999995e-05 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528610+0000", 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0125000000000001e-05 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.583 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:157209 5.5 5:a45a4d12:::benchmark_data_smithi032_22800_object29095:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:11.528945+0000", 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0017192500000000001, 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023462740000000002, 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 157209 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.584 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528945+0000", 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528945+0000", 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528946+0000", 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1179999999999999e-06 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.585 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528951+0000", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0679999999999996e-06 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528963+0000", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2027e-05 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.528971+0000", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4939999999999997e-06 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.586 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529029+0000", 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8458000000000002e-05 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529144+0000", 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000114654 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:11.529256+0000", 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111799 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.587 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:11.588 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 15 2024-04-18T22:24:11.588 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:11.589 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:11.584+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:11.589 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:11.690 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:11.686+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:11.741 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:11.741 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:11.741 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:11.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:11.741 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:11.759 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:11.791 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:11.788+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:11.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:11.890+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:11.960 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.844336691999999, 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:11.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:11.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.844268094, 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:11.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:11.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.971 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.844172248000007, 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.844115365999997, 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.844055694000005, 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.982 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:11.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:11.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843997732000005, 2024-04-18T22:24:11.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:11.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:11.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:11.992 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843778040000004, 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:11.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:11.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:31 3.14 3:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843721110999994, 2024-04-18T22:24:11.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029478286999999999, 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115685+0000", 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:11.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115687+0000", 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7269999999999999e-06 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115721+0000", 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4727000000000001e-05 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115741+0000", 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0061999999999999e-05 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115756+0000", 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4552000000000001e-05 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123023+0000", 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072669270000000003 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124492+0000", 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014694129999999999 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124787+0000", 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029457000000000002 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133893+0000", 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091057510000000005 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144958+0000", 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011065706999999999 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145060+0000", 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000101242 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145163+0000", 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000103609 2024-04-18T22:24:12.003 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:34 3.1c 3:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843625224999997, 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029074662000000001, 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115781+0000", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115782+0000", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2699999999999999e-06 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115790+0000", 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115809+0000", 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9361999999999999e-05 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115823+0000", 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4042e-05 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126612+0000", 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01078895 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126821+0000", 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00020929999999999999 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00028872000000000001 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136558+0000", 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094474940000000007 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144491+0000", 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0079331569999999997 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144751+0000", 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026080599999999998 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144856+0000", 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000104052 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:37 3.13 3:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843562896999998, 2024-04-18T22:24:12.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028256193999999998, 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115843+0000", 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115844+0000", 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.19e-06 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115862+0000", 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7652e-05 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115880+0000", 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8198000000000001e-05 2024-04-18T22:24:12.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115895+0000", 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4459e-05 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121928+0000", 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.006032936 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123149+0000", 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001221176 2024-04-18T22:24:12.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123563+0000", 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00041437000000000001 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132725+0000", 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0091616070000000004 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143428+0000", 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010703285999999999 2024-04-18T22:24:12.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.143525+0000", 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7249000000000004e-05 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144099+0000", 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000574071 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843490419000005, 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:12.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:12.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:12.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843337950999995, 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:20 3.13 3:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.843259227999994, 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028950136000000001, 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.025 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116147+0000", 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116148+0000", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116155+0000", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1960000000000002e-06 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116175+0000", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9851999999999999e-05 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116189+0000", 2024-04-18T22:24:12.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4266e-05 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124278+0000", 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0080891480000000009 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124599+0000", 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032067400000000001 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126170+0000", 2024-04-18T22:24:12.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711119999999999 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136523+0000", 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010352932 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.144849+0000", 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083263859999999999 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145001+0000", 2024-04-18T22:24:12.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000151547 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145097+0000", 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6052999999999997e-05 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.841738371999995, 2024-04-18T22:24:12.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:12.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.035 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 99.84164002, 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:12.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:12.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:12.039 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 66.159918067000007, 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:12.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:12.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:12.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 66.159263854000002, 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:12.047 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 66.159157159000003, 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 66.15729125, 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.052 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:12.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:12.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:12.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 66.156625926999993, 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:12.056 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:12.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:12.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:12.060 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:12.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:12.063 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.056+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:12.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.160+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:12.224 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158863 5.6 5:681782f4:::benchmark_data_smithi032_22800_object30749:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.208450+0000", 2024-04-18T22:24:12.224 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.014905463000000001, 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.014930547000000001, 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158863 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.208450+0000", 2024-04-18T22:24:12.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.208450+0000", 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.208450+0000", 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0600000000000001e-07 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.208453+0000", 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6139999999999999e-06 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.208459+0000", 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2489999999999998e-06 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.208463+0000", 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4689999999999999e-06 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.211493+0000", 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030298780000000002 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.211630+0000", 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013720099999999999 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.211753+0000", 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122471 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.215831+0000", 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0040778860000000002 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223311+0000", 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0074807229999999999 2024-04-18T22:24:12.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158887 5.a 5:5e284048:::benchmark_data_smithi032_22800_object30773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.219785+0000", 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003569703, 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0036813200000000001, 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158887 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219785+0000", 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219785+0000", 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219786+0000", 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0839999999999999e-06 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219791+0000", 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3719999999999998e-06 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219801+0000", 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-05 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219808+0000", 2024-04-18T22:24:12.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9630000000000001e-06 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219867+0000", 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8609000000000001e-05 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219986+0000", 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000119448 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158890 5.0 5:0442a415:::benchmark_data_smithi032_22800_object30776:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.220233+0000", 2024-04-18T22:24:12.235 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0031220800000000002, 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032954030000000001, 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158890 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220233+0000", 2024-04-18T22:24:12.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220233+0000", 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220233+0000", 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5300000000000004e-07 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220237+0000", 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5710000000000002e-06 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220244+0000", 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4059999999999999e-06 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220249+0000", 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4479999999999996e-06 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222181+0000", 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001932013 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222281+0000", 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100436 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222378+0000", 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6824999999999995e-05 2024-04-18T22:24:12.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158891 5.b 5:db0856ea:::benchmark_data_smithi032_22800_object30777:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.220260+0000", 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030955069999999999, 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003335476, 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158891 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220260+0000", 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220260+0000", 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220260+0000", 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5999999999999999e-07 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220263+0000", 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8420000000000001e-06 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220269+0000", 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1229999999999997e-06 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220274+0000", 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9060000000000001e-06 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220314+0000", 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9736e-05 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220394+0000", 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0210000000000006e-05 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158892 5.8 5:13c08f3d:::benchmark_data_smithi032_22800_object30778:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.220283+0000", 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030723780000000002, 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003387143, 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158892 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220283+0000", 2024-04-18T22:24:12.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220283+0000", 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220283+0000", 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8100000000000003e-07 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220286+0000", 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7599999999999998e-06 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220293+0000", 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.686e-06 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220298+0000", 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.614e-06 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220973+0000", 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00067518999999999999 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.221109+0000", 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013580600000000001 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158894 5.2 5:49f1584d:::benchmark_data_smithi032_22800_object30780:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.248 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.220305+0000", 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030500480000000001, 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034245159999999998, 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158894 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220305+0000", 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220305+0000", 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220305+0000", 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0600000000000001e-07 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220307+0000", 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.092e-06 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220313+0000", 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5770000000000001e-06 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220318+0000", 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4360000000000001e-06 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.221938+0000", 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016191420000000001 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222066+0000", 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012853300000000001 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222174+0000", 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107557 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.253 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:158888 5.1 e53/49 5:876da399:::benchmark_data_smithi032_22800_object30774:head v 53'9829, mlcod=53'9827)", 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.220527+0000", 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002828489, 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032674430000000001, 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220527+0000", 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220527+0000", 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220527+0000", 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0500000000000003e-07 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220533+0000", 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4260000000000002e-06 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220544+0000", 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1391e-05 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220550+0000", 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.55e-06 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220615+0000", 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4312000000000004e-05 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220635+0000", 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0387e-05 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:158893 5.5 e53/49 5:ac4992bf:::benchmark_data_smithi032_22800_object30779:head v 53'9792, mlcod=53'9790)", 2024-04-18T22:24:12.257 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.220588+0000", 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002767224, 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032616279999999999, 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220588+0000", 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220588+0000", 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220588+0000", 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8299999999999998e-07 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220592+0000", 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0620000000000002e-06 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220600+0000", 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0350000000000001e-06 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220606+0000", 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.463e-06 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220652+0000", 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6703000000000003e-05 2024-04-18T22:24:12.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220673+0000", 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0559000000000001e-05 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:158889 5.1 e53/49 5:8bd166fd:::benchmark_data_smithi032_22800_object30775:head v 53'9830, mlcod=53'9827)", 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.222345+0000", 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010099779999999999, 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001571322, 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222345+0000", 2024-04-18T22:24:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222345+0000", 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222346+0000", 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5499999999999998e-07 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222351+0000", 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9200000000000003e-06 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222359+0000", 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6759999999999996e-06 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222365+0000", 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0870000000000003e-06 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222415+0000", 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9472000000000003e-05 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222437+0000", 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1780000000000002e-05 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(5.0 53/49 [PushReplyOp(5:0346f88a:::benchmark_data_smithi032_22800_object69556:head)])", 2024-04-18T22:24:12.265 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.222428+0000", 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000927187, 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015397379999999999, 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222428+0000", 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222428+0000", 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222428+0000", 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8700000000000002e-07 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222431+0000", 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3790000000000002e-06 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222438+0000", 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6030000000000001e-06 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222444+0000", 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2280000000000003e-06 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158895 5.3 5:ccea156c:::benchmark_data_smithi032_22800_object30781:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.222554+0000", 2024-04-18T22:24:12.268 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00080094400000000005, 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001457355, 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158895 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222554+0000", 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222554+0000", 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222555+0000", 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9500000000000001e-07 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222559+0000", 2024-04-18T22:24:12.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6550000000000002e-06 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222571+0000", 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2064e-05 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222577+0000", 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9889999999999997e-06 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222952+0000", 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00037497099999999998 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223080+0000", 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012803399999999999 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223195+0000", 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115463 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.272 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158896 5.4 5:2a1737d9:::benchmark_data_smithi032_22800_object30782:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.222592+0000", 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00076330099999999998, 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015173999999999999, 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158896 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222592+0000", 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222592+0000", 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222592+0000", 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1100000000000001e-07 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222594+0000", 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2189999999999998e-06 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222602+0000", 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.391e-06 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222606+0000", 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6450000000000004e-06 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222654+0000", 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7649999999999999e-05 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.222774+0000", 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000119521 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158899 5.3 5.f7a1f3a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.223890+0000", 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.999465, 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027786599999999999, 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158899 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223890+0000", 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223890+0000", 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223891+0000", 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1900000000000001e-07 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223895+0000", 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3359999999999997e-06 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223902+0000", 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6150000000000002e-06 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.223910+0000", 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3380000000000005e-06 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.280 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158901 5.8 5.f5318cd8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.224190+0000", 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9991651, 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4372000000000002e-05, 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.281 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158901 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224190+0000", 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224190+0000", 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224191+0000", 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8000000000000005e-07 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224196+0000", 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8919999999999999e-06 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224205+0000", 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9190000000000005e-06 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224212+0000", 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1890000000000001e-06 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:158897 5.1 e53/49)", 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.224196+0000", 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9991593, 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8182999999999993e-05, 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224196+0000", 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224196+0000", 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224197+0000", 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.44e-07 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224204+0000", 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.875e-06 2024-04-18T22:24:12.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224214+0000", 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0093e-05 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224222+0000", 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4800000000000004e-06 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.287 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158881 5.e 5:7f17bd96:::benchmark_data_smithi032_22800_object30767:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.216163+0000", 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0071921470000000003, 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0081604390000000002, 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158881 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216163+0000", 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216163+0000", 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216164+0000", 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.85e-07 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216168+0000", 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9580000000000001e-06 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216177+0000", 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6989999999999998e-06 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216184+0000", 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.708e-06 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.219895+0000", 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037113570000000002 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.220010+0000", 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114595 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.224290+0000", 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004280313 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.293 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:158884 5.8 5:1a8b5e8f:::benchmark_data_smithi032_22800_object30770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:12.293 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:12.216256+0000", 2024-04-18T22:24:12.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0070993059999999997, 2024-04-18T22:24:12.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0081337449999999995, 2024-04-18T22:24:12.293 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 158884 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216256+0000", 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216256+0000", 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216256+0000", 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216258+0000", 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9190000000000002e-06 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216264+0000", 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5910000000000003e-06 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.216275+0000", 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1416999999999999e-05 2024-04-18T22:24:12.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.218096+0000", 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018207620000000001 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:12.218214+0000", 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011825 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 17 2024-04-18T22:24:12.298 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:12.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:12.301 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:12.298+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:12.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:12.398+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:12.499 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:12.499 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:12.499 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:12.500 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:12.500 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:12.500 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.496+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:12.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:12.606 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.604+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:12.705 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.700+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:12.715 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.915756864000002, 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:12.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:12.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:12.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:12.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:12.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:12.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:12.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:12.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.724 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:12.725 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.914100485000006, 2024-04-18T22:24:12.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:12.725 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.725 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.726 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.726 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:12.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.726 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:12.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:12.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:12.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:12.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:12.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:12.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:12.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:12.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:12.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:12.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:12.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:12.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:12.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:12.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:12.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.913975768, 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.737 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:12.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:12.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:12.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.912196109999996, 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:12.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:12.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.911713438999996, 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.749 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.911474364, 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.756 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.757 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.758 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.759 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:12.760 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.762 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.911353593000001, 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:12.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.764 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.765 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:12.768 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.908464180999999, 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.769 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:12.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.771 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.772 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:12.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:12.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.908342361999999, 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:12.775 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:12.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.781 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.908231912999995, 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.782 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:12.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.908162445000002, 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.788 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:12.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:12.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:12.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.793 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 66.908105304000003, 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:12.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:12.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:12.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:12.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:12.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.678929793, 2024-04-18T22:24:12.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:12.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:12.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.6742867940000004, 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:12.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:12.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:12.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.6651890569999992, 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:12.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.6561691320000005, 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.821 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:12.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.825 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:12.828 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.6401577639999996, 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:12.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.835 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.6373543440000002, 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.836 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:12.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:12.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137384 5.0 5:0d4b5451:::benchmark_data_smithi032_22800_object9270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:12.842 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.5919777760000002, 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.054059477000000002, 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137384 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:12.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122526+0000", 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122528+0000", 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1629999999999999e-06 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122533+0000", 2024-04-18T22:24:12.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5570000000000003e-06 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122554+0000", 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1222000000000001e-05 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.122571+0000", 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7005000000000001e-05 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156535+0000", 2024-04-18T22:24:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.033963356 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.156833+0000", 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00029849800000000001 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.157060+0000", 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022695300000000001 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175844+0000", 2024-04-18T22:24:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018784072999999998 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176295+0000", 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00045056399999999998 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176394+0000", 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9667000000000006e-05 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176586+0000", 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019141899999999999 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137394 5.0 5:082322e3:::benchmark_data_smithi032_22800_object9280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.5816280389999999, 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.050257665, 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:12.849 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137394 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132876+0000", 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132878+0000", 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6169999999999999e-06 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132885+0000", 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.126e-06 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132905+0000", 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0369000000000001e-05 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.132921+0000", 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5699999999999999e-05 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172761+0000", 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.039840484000000002 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172994+0000", 2024-04-18T22:24:12.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00023306499999999999 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.173221+0000", 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000226949 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.179433+0000", 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062120439999999999 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:12.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182890+0000", 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034568089999999999 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.182980+0000", 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9844000000000002e-05 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:12.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.183134+0000", 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015365800000000001 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:12.855 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:12.857 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:12.858 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.852+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:12.958 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:12.952+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:13.014 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:13.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:160820 5.d 5:b0941693:::benchmark_data_smithi032_22800_object32706:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.010617+0000", 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0035666809999999999, 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0035907249999999999, 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 160820 2024-04-18T22:24:13.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010617+0000", 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010617+0000", 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010618+0000", 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010621+0000", 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9380000000000001e-06 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010629+0000", 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2509999999999994e-06 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010634+0000", 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1390000000000003e-06 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010686+0000", 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2385000000000001e-05 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010824+0000", 2024-04-18T22:24:13.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013782099999999999 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:160825 5.d 5:bd7dc419:::benchmark_data_smithi032_22800_object32711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.012005+0000", 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0021784360000000002, 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022734980000000001, 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:13.019 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 160825 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012005+0000", 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012005+0000", 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012006+0000", 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4499999999999996e-07 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012010+0000", 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3869999999999997e-06 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012022+0000", 2024-04-18T22:24:13.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1292e-05 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012029+0000", 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3359999999999997e-06 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012419+0000", 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038947199999999997 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012568+0000", 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000149541 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:160826 5.d 5.9030616d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.012040+0000", 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0021432059999999999, 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022990139999999998, 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:13.023 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 160826 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012040+0000", 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012040+0000", 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012041+0000", 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2300000000000002e-07 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012043+0000", 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3e-06 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012050+0000", 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1659999999999997e-06 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012055+0000", 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.775e-06 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.026 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:160828 5.2 e53/23 5:4548f195:::benchmark_data_smithi032_22800_object32714:head v 53'10085, mlcod=53'10083)", 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.013115+0000", 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001068587, 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012769140000000001, 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013115+0000", 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013115+0000", 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013116+0000", 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1399999999999997e-07 2024-04-18T22:24:13.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013119+0000", 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8360000000000002e-06 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013127+0000", 2024-04-18T22:24:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6170000000000001e-06 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013132+0000", 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.639e-06 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013178+0000", 2024-04-18T22:24:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6752000000000002e-05 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013190+0000", 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1822e-05 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:160829 5.c e53/19 5:3e73b5e6:::benchmark_data_smithi032_22800_object32715:head v 53'10083, mlcod=53'10081)", 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.013149+0000", 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001034395, 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012997219999999999, 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.031 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013149+0000", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013149+0000", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013150+0000", 2024-04-18T22:24:13.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8799999999999998e-07 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013153+0000", 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0900000000000001e-06 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013158+0000", 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3589999999999999e-06 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013162+0000", 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0759999999999996e-06 2024-04-18T22:24:13.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013206+0000", 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4397000000000002e-05 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.013226+0000", 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9545e-05 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop_reply(client.4586.0:160814 5.5 e53/49)", 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.014349+0000", 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9998341, 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015204199999999999, 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014349+0000", 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014349+0000", 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014350+0000", 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4999999999999998e-07 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014353+0000", 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7709999999999998e-06 2024-04-18T22:24:13.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014361+0000", 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0090000000000007e-06 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014365+0000", 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1350000000000002e-06 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop_reply(client.4586.0:160817 5.5 e53/49)", 2024-04-18T22:24:13.037 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.014400+0000", 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9997835, 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000144578, 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014400+0000", 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014400+0000", 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014401+0000", 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1699999999999999e-07 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014402+0000", 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7710000000000001e-06 2024-04-18T22:24:13.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014409+0000", 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1990000000000003e-06 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014412+0000", 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.2449999999999998e-06 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.040 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop_reply(client.4586.0:160818 5.5 e53/49)", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.014418+0000", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9997659, 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016866599999999999, 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014418+0000", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014418+0000", 2024-04-18T22:24:13.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014418+0000", 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.24e-07 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014419+0000", 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.212e-06 2024-04-18T22:24:13.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014422+0000", 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.965e-06 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014424+0000", 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.328e-06 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.043 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDPGPush(5.0 53/49 [PushOp(5:0395b28a:::benchmark_data_smithi032_22800_object123978:head, version: 45'7651, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5:0395b28a:::benchmark_data_smithi032_22800_object123978:head@45'7651, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.014427+0000", 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9997563, 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000231633, 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014427+0000", 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014427+0000", 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014428+0000", 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0999999999999994e-08 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014484+0000", 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6677e-05 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014498+0000", 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3638000000000001e-05 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014508+0000", 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0195e-05 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014590+0000", 2024-04-18T22:24:13.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1755999999999994e-05 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014601+0000", 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1599e-05 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "pg_backfill(progress 5.0 e 53/53 lb 5:0395839e:::benchmark_data_smithi032_22800_object86554:head)", 2024-04-18T22:24:13.047 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.014524+0000", 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9996595, 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000189211, 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014524+0000", 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014524+0000", 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014524+0000", 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6100000000000002e-07 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014529+0000", 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0080000000000002e-06 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014540+0000", 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0856000000000001e-05 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.014544+0000", 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8979999999999998e-06 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:160814 5.5 5:a8ec3adf:::benchmark_data_smithi032_22800_object32700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.006441+0000", 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0077425899999999997, 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083224800000000002, 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 160814 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006441+0000", 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006441+0000", 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006442+0000", 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8100000000000002e-07 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006446+0000", 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8940000000000003e-06 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006453+0000", 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7849999999999992e-06 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006458+0000", 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9559999999999996e-06 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006497+0000", 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8364000000000001e-05 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006571+0000", 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4327999999999997e-05 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.006636+0000", 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4727000000000006e-05 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010783+0000", 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.004147408 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:160817 5.5 5:ad571c4f:::benchmark_data_smithi032_22800_object32703:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.007642+0000", 2024-04-18T22:24:13.055 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0065420959999999998, 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0071986740000000004, 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 160817 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007642+0000", 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007642+0000", 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007642+0000", 2024-04-18T22:24:13.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1600000000000002e-07 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007644+0000", 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4789999999999999e-06 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007650+0000", 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3120000000000001e-06 2024-04-18T22:24:13.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007654+0000", 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3880000000000002e-06 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007997+0000", 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00034340599999999998 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.008078+0000", 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0294000000000001e-05 2024-04-18T22:24:13.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.008149+0000", 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1606000000000005e-05 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.010833+0000", 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026832959999999999 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.060 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:160818 5.5 5:ad78b8c2:::benchmark_data_smithi032_22800_object32704:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.007677+0000", 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0065063760000000003, 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072365729999999996, 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 160818 2024-04-18T22:24:13.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007677+0000", 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007677+0000", 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:13.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007678+0000", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9500000000000003e-07 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007680+0000", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1019999999999999e-06 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007695+0000", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5446000000000001e-05 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.007702+0000", 2024-04-18T22:24:13.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9800000000000001e-06 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.009135+0000", 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014332069999999999 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.009211+0000", 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5949000000000001e-05 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.009280+0000", 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8775000000000005e-05 2024-04-18T22:24:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.012397+0000", 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0031167970000000001 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 13 2024-04-18T22:24:13.065 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:13.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:24:13.068 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:13.068 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.066+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:13.168 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.166+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:13.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:13.268 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:13.268 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:13.268 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:13.268 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:13.268 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:13.269 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.266+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:13.288 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:13.369 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:13.364+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:13.373 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:13.396 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:13.399 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:13.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:13.470 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.466+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:13.490 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:13.490 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:13.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:13.571 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.570+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:13.667 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:13.667 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:13.672 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:13.668+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:13.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:13.775 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:13.772+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:13.874 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.870+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:13.885 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:13.885 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:13.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:13.974 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:13.970+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:24:14.075 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.074+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.107 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:14.107 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:14.130 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:14.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.174+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.274+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.279 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:14.280 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:14.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:14.377 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:14.372+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:14.480 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.474+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.504 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:14.504 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:14.525 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:14.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.610+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.678+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.723 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:14.723 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:14.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:14.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.782+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:14.883 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:14.880+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:14.925 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:14.925 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:14.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:14.989 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:14.982+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:15.084 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.080+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.112 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:15.113 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:15.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:15.222 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.216+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~41s 2024-04-18T22:24:15.287 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.280+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.323 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:15.323 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:15.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:15.391 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.388+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:15.487 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.484+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.498 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:15.498 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:15.518 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:15.558 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:15.577 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:15.582 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:15.583 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:15.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:15.586+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:15.672 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:15.672 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:15.672 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:15.672 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:15.672 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:15.689 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.684+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.690 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:15.790 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:15.784+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:7 3.10 3:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:15.885 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.769268825, 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032467541000000003, 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115069+0000", 2024-04-18T22:24:15.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115071+0000", 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.59e-06 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115080+0000", 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9430000000000006e-06 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115099+0000", 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9185e-05 2024-04-18T22:24:15.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115115+0000", 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5441999999999999e-05 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123314+0000", 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0081997790000000008 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123522+0000", 2024-04-18T22:24:15.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000207357 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.123913+0000", 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000391319 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.135331+0000", 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011418397 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147072+0000", 2024-04-18T22:24:15.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011740802999999999 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147453+0000", 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038094000000000002 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.147537+0000", 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3786e-05 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.769200227, 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:15.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:15.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:15.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.76910438100001, 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:15.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.901 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.769047499, 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.768987827, 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:15.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.768929865, 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:15.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:15.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:15.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.768710173, 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:15.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:15.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.768422552, 2024-04-18T22:24:15.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:15.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:15.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.76827008399999, 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:15.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:15.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.76667050499999, 2024-04-18T22:24:15.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:15.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:15.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:15.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 103.766572153, 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:15.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:15.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 70.084850200000005, 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:15.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:15.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:15.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 70.084195987000001, 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:15.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:15.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 70.084089292000002, 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:15.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 70.082223382999999, 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:15.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:15.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 70.081558060000006, 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:15.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.5397423290000001, 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:15.964 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:15.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:15.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:15.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161664 5.5 5:ae49fe32:::benchmark_data_smithi032_22800_object33550:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.346182+0000", 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.5381562419999999, 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.031077779, 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.970 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161664 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346182+0000", 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346182+0000", 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346183+0000", 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7199999999999997e-07 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346187+0000", 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6789999999999998e-06 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346197+0000", 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0601000000000001e-05 2024-04-18T22:24:15.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346204+0000", 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7120000000000003e-06 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346383+0000", 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017960400000000001 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346609+0000", 2024-04-18T22:24:15.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022539000000000001 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.346721+0000", 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112149 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.348587+0000", 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001865675 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.974 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377108+0000", 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.028521449000000001 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377160+0000", 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1631000000000001e-05 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377260+0000", 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100017 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.975 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.533601832, 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:15.976 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:15.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.981 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:161678 5.7 e53/23 5:e1c8ea11:::benchmark_data_smithi032_22800_object33564:head v 53'10195, mlcod=53'10192)", 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.353147+0000", 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.5311908490000001, 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.030701836, 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.353147+0000", 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:15.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.353147+0000", 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.353148+0000", 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7999999999999996e-07 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.353155+0000", 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8929999999999999e-06 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.353163+0000", 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0919999999999998e-06 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.353169+0000", 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1229999999999997e-06 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.378486+0000", 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.025316722999999999 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.378506+0000", 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9831e-05 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383808+0000", 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005302485 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383849+0000", 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1208999999999998e-05 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.985 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:15.986 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:15.986 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:15.987 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:15.986+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:15.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:16.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:24:16.088 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:16.084+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:16.145 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:16.145 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167970 5.b 5:d3feca71:::benchmark_data_smithi032_22800_object39856:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.138250+0000", 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0064738, 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064979019999999998, 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167970 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138250+0000", 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138250+0000", 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138250+0000", 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8800000000000002e-07 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138254+0000", 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0230000000000003e-06 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138271+0000", 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7037999999999998e-05 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138278+0000", 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5599999999999999e-06 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138325+0000", 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7252000000000001e-05 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138442+0000", 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116952 2024-04-18T22:24:16.149 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167972 5.8 5:1902f831:::benchmark_data_smithi032_22800_object39858:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.138623+0000", 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.006100566, 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062011119999999999, 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.150 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167972 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138623+0000", 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138623+0000", 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138624+0000", 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0500000000000003e-07 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138627+0000", 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.878e-06 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138634+0000", 2024-04-18T22:24:16.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.55e-06 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.138640+0000", 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8340000000000004e-06 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140201+0000", 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0015610089999999999 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140324+0000", 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122846 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167977 5.0 5:0f317794:::benchmark_data_smithi032_22800_object39863:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.140607+0000", 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0041164000000000001, 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004298138, 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:16.154 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167977 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140607+0000", 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140607+0000", 2024-04-18T22:24:16.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140608+0000", 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3499999999999996e-07 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140613+0000", 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4729999999999999e-06 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140622+0000", 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4200000000000007e-06 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.140627+0000", 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.733e-06 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.142883+0000", 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022553349999999998 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143025+0000", 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014240800000000001 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143160+0000", 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013484599999999999 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.158 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167979 5.3 5:c515880c:::benchmark_data_smithi032_22800_object39865:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.141509+0000", 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032140900000000002, 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034626919999999999, 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167979 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141509+0000", 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141509+0000", 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141510+0000", 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1699999999999998e-07 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141513+0000", 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3830000000000001e-06 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141522+0000", 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7819999999999999e-06 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141527+0000", 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6460000000000001e-06 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141578+0000", 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1672000000000003e-05 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141720+0000", 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014200199999999999 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141837+0000", 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116789 2024-04-18T22:24:16.162 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167980 5.0 5.7f214130 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.141540+0000", 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0031832940000000001, 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035047450000000001, 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.163 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167980 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141540+0000", 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141540+0000", 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141540+0000", 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1600000000000002e-07 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141542+0000", 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141548+0000", 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.169e-06 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141551+0000", 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.999e-06 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167981 5.8 5.20e990c8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.141652+0000", 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030711760000000001, 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003457534, 2024-04-18T22:24:16.166 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167981 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141652+0000", 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141652+0000", 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141653+0000", 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8999999999999997e-07 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141656+0000", 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8679999999999999e-06 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141661+0000", 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.852e-06 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.141665+0000", 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2799999999999999e-06 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167983 5.4 5:22a40415:::benchmark_data_smithi032_22800_object39869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.169 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.143246+0000", 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014774670000000001, 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019144089999999999, 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167983 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143246+0000", 2024-04-18T22:24:16.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143246+0000", 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143246+0000", 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6499999999999999e-07 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143250+0000", 2024-04-18T22:24:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.36e-06 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143256+0000", 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3589999999999996e-06 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143261+0000", 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7910000000000004e-06 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143313+0000", 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1545999999999997e-05 2024-04-18T22:24:16.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143390+0000", 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6976000000000003e-05 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167984 5.3 5:cd08934a:::benchmark_data_smithi032_22800_object39870:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.143283+0000", 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0014402659999999999, 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001947182, 2024-04-18T22:24:16.173 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167984 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143283+0000", 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143283+0000", 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143284+0000", 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4099999999999999e-07 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143286+0000", 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0109999999999999e-06 2024-04-18T22:24:16.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143293+0000", 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2010000000000002e-06 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143296+0000", 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.45e-06 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.176 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143683+0000", 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00038727500000000002 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143822+0000", 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013894299999999999 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.143962+0000", 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013926699999999999 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167985 5.6 5:6b83a067:::benchmark_data_smithi032_22800_object39871:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144126+0000", 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00059757300000000001, 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011808350000000001, 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167985 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.178 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144126+0000", 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144126+0000", 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144127+0000", 2024-04-18T22:24:16.179 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7700000000000003e-07 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144131+0000", 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5580000000000002e-06 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144143+0000", 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1321999999999999e-05 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.180 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144150+0000", 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9709999999999998e-06 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144202+0000", 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.274e-05 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144366+0000", 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000163466 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144469+0000", 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103423 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167987 5.3 5.606678e3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144166+0000", 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000557338, 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001212744, 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167987 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144166+0000", 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144166+0000", 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144167+0000", 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0299999999999999e-07 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144170+0000", 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0199999999999999e-06 2024-04-18T22:24:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144178+0000", 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1009999999999992e-06 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144184+0000", 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1249999999999998e-06 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167988 5.3 5.eadf7a53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144195+0000", 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00052859499999999998, 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001238376, 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167988 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144195+0000", 2024-04-18T22:24:16.186 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144195+0000", 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144195+0000", 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4000000000000001e-07 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144198+0000", 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3360000000000002e-06 2024-04-18T22:24:16.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144204+0000", 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8540000000000002e-06 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144209+0000", 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0180000000000001e-06 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167989 5.c 5:35a7f0d5:::benchmark_data_smithi032_22800_object39875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.188 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144407+0000", 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00031663900000000002, 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010972860000000001, 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167989 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144407+0000", 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144407+0000", 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144408+0000", 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.048e-06 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144412+0000", 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2760000000000002e-06 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144421+0000", 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7280000000000001e-06 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144427+0000", 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6950000000000003e-06 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144518+0000", 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0160000000000004e-05 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144628+0000", 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110455 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144724+0000", 2024-04-18T22:24:16.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6216000000000002e-05 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167990 5.8 5.16ed8318 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144442+0000", 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00028169399999999998, 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001126351, 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167990 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144442+0000", 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144442+0000", 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144442+0000", 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6400000000000003e-07 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144445+0000", 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5170000000000002e-06 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144451+0000", 2024-04-18T22:24:16.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2230000000000004e-06 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144456+0000", 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3179999999999997e-06 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167991 5.8 5.5e873418 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144646+0000", 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.7010999999999998e-05, 2024-04-18T22:24:16.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00096940100000000005, 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167991 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144646+0000", 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144646+0000", 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144647+0000", 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7300000000000005e-07 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144651+0000", 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.737e-06 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144659+0000", 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3950000000000001e-06 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144666+0000", 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8700000000000003e-06 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167992 5.b 5.ca8c400b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.144740+0000", 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999833, 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00092456799999999996, 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167992 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144740+0000", 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144740+0000", 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144740+0000", 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5000000000000003e-07 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144743+0000", 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.672e-06 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144764+0000", 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0644000000000001e-05 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.144769+0000", 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5890000000000002e-06 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(5.0 54/49 [PushReplyOp(5:0521f404:::benchmark_data_smithi032_22800_object72248:head)])", 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.145078+0000", 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9996457, 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00063972900000000001, 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145078+0000", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145078+0000", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145078+0000", 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7599999999999998e-07 2024-04-18T22:24:16.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145085+0000", 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1779999999999997e-06 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145094+0000", 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7520000000000002e-06 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145108+0000", 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4499e-05 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:167977 5.0 e54/49)", 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:16.145469+0000", 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9992547, 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000289463, 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.206 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145469+0000", 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145469+0000", 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145469+0000", 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4200000000000002e-07 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145471+0000", 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3910000000000001e-06 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145477+0000", 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3329999999999996e-06 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:16.145480+0000", 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6899999999999998e-06 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.209 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:16.209 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 17 2024-04-18T22:24:16.209 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:16.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:16.211 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:16.204+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:16.311 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:16.310+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:16.406 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:16.406 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:16.406 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:16.407 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:16.407 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:16.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:16.410+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:16.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:16.513 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:16.508+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.779376729999996, 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.777720350999999, 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.777595633999994, 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:16.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:16.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.775815976000004, 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:16.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:16.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:16.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.602 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.775333305000004, 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:16.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:16.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:16.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:16.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:16.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.775094229999993, 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.608 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.613 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.774973458999995, 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.772084047000007, 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.619 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:16.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.624 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.771962228000007, 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:16.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:16.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.630 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.771851779000002, 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:16.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:16.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:16.636 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.771782310999996, 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:16.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:16.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:16.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:16.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 70.771725169999996, 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:16.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:16.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:16.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.542549659000001, 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:16.647 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:16.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:16.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:16.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:16.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.537906660000001, 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:16.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:16.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:16.659 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.528808923, 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:16.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:16.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:16.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.519788997999999, 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:16.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:16.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:16.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:16.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.50377763, 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:16.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.500974210000001, 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.679 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:16.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167496 5.8 5:15a92cd7:::benchmark_data_smithi032_22800_object39382:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.7094992, 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073409221999999996, 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167496 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:16.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868626+0000", 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868629+0000", 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6619999999999998e-06 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868636+0000", 2024-04-18T22:24:16.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.002e-06 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868642+0000", 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5079999999999996e-06 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868675+0000", 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3651e-05 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868752+0000", 2024-04-18T22:24:16.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6422999999999997e-05 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941860+0000", 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073108464999999997 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941927+0000", 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6571999999999999e-05 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.942034+0000", 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010723 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167499 5.0 5:02d6841a:::benchmark_data_smithi032_22800_object39385:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.70887367700000004, 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.074657837000000005, 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167499 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:16.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869261+0000", 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869264+0000", 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1719999999999999e-06 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869269+0000", 2024-04-18T22:24:16.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8910000000000003e-06 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869274+0000", 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6959999999999996e-06 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870422+0000", 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00114858 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870532+0000", 2024-04-18T22:24:16.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010966199999999999 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870646+0000", 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011456 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.937008+0000", 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.066361693999999999 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:16.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943734+0000", 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067261580000000003 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943815+0000", 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1174000000000006e-05 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943908+0000", 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2709999999999998e-05 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:16.696 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:16.696 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:16.697 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:16.698 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:16.692+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:16.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:16.794+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:16.854 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:16.854 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:169625 5.7 e54/23 5:ea2efd9f:::benchmark_data_smithi032_22800_object41511:head v 54'10708, mlcod=54'10705)", 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.851528+0000", 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0034507140000000001, 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034760659999999999, 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.851528+0000", 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.851528+0000", 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.851528+0000", 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7800000000000001e-07 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.851535+0000", 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0879999999999997e-06 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.851546+0000", 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0847e-05 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.851551+0000", 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5390000000000001e-06 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.852364+0000", 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000813742 2024-04-18T22:24:16.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.852384+0000", 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9462000000000001e-05 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:169630 5.7 e54/23 5:eadcf5c8:::benchmark_data_smithi032_22800_object41516:head v 54'10709, mlcod=54'10705)", 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.853480+0000", 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0014978459999999999, 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015713459999999999, 2024-04-18T22:24:16.858 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853480+0000", 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853480+0000", 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853481+0000", 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0399999999999996e-07 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853487+0000", 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8850000000000004e-06 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853498+0000", 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0759e-05 2024-04-18T22:24:16.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853506+0000", 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4880000000000007e-06 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854130+0000", 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00062342500000000004 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854147+0000", 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7495000000000001e-05 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.861 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "pg_backfill(progress 5.0 e 54/54 lb 5:0579cc2c:::benchmark_data_smithi032_22800_object90328:head)", 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.853749+0000", 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001229302, 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001349648, 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853749+0000", 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853749+0000", 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853749+0000", 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8599999999999999e-07 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853755+0000", 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.879e-06 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853761+0000", 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4349999999999996e-06 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853764+0000", 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.715e-06 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854443+0000", 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00067836299999999997 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854458+0000", 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5259000000000001e-05 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:169632 5.5 5:ae8922de:::benchmark_data_smithi032_22800_object41518:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.853824+0000", 2024-04-18T22:24:16.865 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001153987, 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001308046, 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 169632 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853824+0000", 2024-04-18T22:24:16.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853824+0000", 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853825+0000", 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8599999999999997e-07 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853828+0000", 2024-04-18T22:24:16.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.9289999999999998e-06 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853835+0000", 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7519999999999999e-06 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853839+0000", 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1749999999999998e-06 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853879+0000", 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0030999999999997e-05 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.853999+0000", 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012037800000000001 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854145+0000", 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014557799999999999 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:169629 5.0 e54/49)", 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.854019+0000", 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00095924400000000005, 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001152705, 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854019+0000", 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854019+0000", 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854020+0000", 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4000000000000002e-07 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854024+0000", 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7419999999999997e-06 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854033+0000", 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1160000000000005e-06 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854041+0000", 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5089999999999996e-06 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:169631 5.2 e54/23 5:451a2167:::benchmark_data_smithi032_22800_object41517:head v 54'10614, mlcod=54'10611)", 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.854079+0000", 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00089890800000000002, 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001117306, 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854079+0000", 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854079+0000", 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854080+0000", 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5499999999999999e-07 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854084+0000", 2024-04-18T22:24:16.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4220000000000002e-06 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854093+0000", 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2439999999999999e-06 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854099+0000", 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3129999999999998e-06 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854141+0000", 2024-04-18T22:24:16.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1863000000000002e-05 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854156+0000", 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5122e-05 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:169634 5.9 5:93c6ed13:::benchmark_data_smithi032_22800_object41520:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.854413+0000", 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00056571400000000004, 2024-04-18T22:24:16.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00081521199999999995, 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 169634 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854413+0000", 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854413+0000", 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854414+0000", 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0160000000000001e-06 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854418+0000", 2024-04-18T22:24:16.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0160000000000002e-06 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854428+0000", 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0715e-05 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854436+0000", 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0250000000000002e-06 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854550+0000", 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011330799999999999 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854678+0000", 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012815299999999999 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:169635 5.6 e54/19 5:6a52f7c6:::benchmark_data_smithi032_22800_object41521:head v 54'10507, mlcod=54'10505)", 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.854862+0000", 2024-04-18T22:24:16.880 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000116066, 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000420491, 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854862+0000", 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854862+0000", 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854863+0000", 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.44e-07 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854868+0000", 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3800000000000002e-06 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854878+0000", 2024-04-18T22:24:16.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0071999999999999e-05 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854885+0000", 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3829999999999997e-06 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854946+0000", 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1554000000000001e-05 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.854961+0000", 2024-04-18T22:24:16.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4572e-05 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:169636 5.5 5.b2aeb045 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:16.855039+0000", 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.999939, 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030703000000000002, 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:16.884 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 169636 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.855039+0000", 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.855039+0000", 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.855040+0000", 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9599999999999998e-07 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.855044+0000", 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0500000000000002e-06 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.855056+0000", 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1582e-05 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:16.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:16.855062+0000", 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7730000000000002e-06 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:24:16.887 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:16.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:16.898 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:16.892+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:16.999 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:16.998+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:17.051 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:24:17.083 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:17.083 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:17.083 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:17.083 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:17.083 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:17.099 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:17.096+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:17.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:17.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:17.198+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.180803508, 2024-04-18T22:24:17.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:17.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:17.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:17.302 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.180707662, 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:17.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:17.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.18065077999999, 2024-04-18T22:24:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:17.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:17.313 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.180591108, 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:17.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:17.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:17.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.318 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:17.319 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.180533146, 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:17.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:17.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.180313454, 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:17.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:17.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.180025833, 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.330 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:17.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:17.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:17.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:17.335 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.17987336500001, 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:17.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.17827378600001, 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.341 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.344 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:17.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 105.178175434, 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.346 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.496453481000003, 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.350 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:17.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.495799267999999, 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:17.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.495692572999999, 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.359 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:17.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:17.363 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.493826663999997, 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:17.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 71.493161341000004, 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:17.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 3.9513456099999997, 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.372 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:17.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 3.9452051130000001, 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.4288990319999999, 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.427365108, 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.389 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:17.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:17.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 1.421539753, 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.395 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:17.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:17.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:17.400 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:17.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:17.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:17.402+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:17.502 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:17.496+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171316 5.2 5:444c0707:::benchmark_data_smithi032_22800_object43202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.558025+0000", 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00078688499999999997, 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00080511099999999996, 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171316 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558025+0000", 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558025+0000", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558026+0000", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558029+0000", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9390000000000002e-06 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558036+0000", 2024-04-18T22:24:17.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4569999999999999e-06 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558042+0000", 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4689999999999996e-06 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558082+0000", 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0450000000000001e-05 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558182+0000", 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9328999999999997e-05 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558281+0000", 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9135999999999994e-05 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171317 5.3 5:cd42123b:::benchmark_data_smithi032_22800_object43203:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.558051+0000", 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00076125699999999999, 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00082472899999999996, 2024-04-18T22:24:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171317 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558051+0000", 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558051+0000", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558051+0000", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6800000000000001e-07 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558054+0000", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5610000000000001e-06 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558059+0000", 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558062+0000", 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9809999999999999e-06 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558102+0000", 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9691999999999999e-05 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558213+0000", 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111106 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558318+0000", 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104375 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171318 5.0 5:05077f1d:::benchmark_data_smithi032_22800_object43204:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.558072+0000", 2024-04-18T22:24:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00074058100000000003, 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000844749, 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171318 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558072+0000", 2024-04-18T22:24:17.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558072+0000", 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558072+0000", 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.34e-07 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558075+0000", 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0060000000000001e-06 2024-04-18T22:24:17.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558082+0000", 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7510000000000002e-06 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558087+0000", 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4330000000000003e-06 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558129+0000", 2024-04-18T22:24:17.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1947999999999999e-05 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558241+0000", 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112169 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.558420+0000", 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000178631 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.571 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171301 5.4 5:2140b390:::benchmark_data_smithi032_22800_object43187:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.553370+0000", 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.005442133, 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055859250000000003, 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171301 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553370+0000", 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553370+0000", 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553371+0000", 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4900000000000005e-07 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553375+0000", 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8879999999999999e-06 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553382+0000", 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0210000000000002e-06 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553387+0000", 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7539999999999997e-06 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553422+0000", 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5519999999999999e-05 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553546+0000", 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124332 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171303 5.8 5:17919722:::benchmark_data_smithi032_22800_object43189:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.553490+0000", 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0053218969999999999, 2024-04-18T22:24:17.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055012220000000001, 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171303 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553490+0000", 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553490+0000", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553491+0000", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6400000000000003e-07 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553494+0000", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7549999999999999e-06 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553500+0000", 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5119999999999997e-06 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553506+0000", 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4729999999999999e-06 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553535+0000", 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9879e-05 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553643+0000", 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107482 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171307 5.b 5:dd53a86a:::benchmark_data_smithi032_22800_object43193:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.553517+0000", 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0052948939999999996, 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005508438, 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.579 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171307 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553517+0000", 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553517+0000", 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553518+0000", 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3799999999999998e-07 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553520+0000", 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.283e-06 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553526+0000", 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2260000000000002e-06 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553532+0000", 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.259e-06 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553568+0000", 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6207000000000001e-05 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.553677+0000", 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010883300000000001 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.582 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171309 5.2 5:480a1429:::benchmark_data_smithi032_22800_object43195:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.555550+0000", 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032623690000000002, 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035095840000000001, 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171309 2024-04-18T22:24:17.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555550+0000", 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555550+0000", 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555551+0000", 2024-04-18T22:24:17.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6199999999999996e-07 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555555+0000", 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3479999999999997e-06 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555565+0000", 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5319999999999995e-06 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555570+0000", 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7479999999999999e-06 2024-04-18T22:24:17.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555608+0000", 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7224000000000001e-05 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555730+0000", 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122347 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555842+0000", 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112062 2024-04-18T22:24:17.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557991+0000", 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021491230000000002 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171310 5.4 5:2e6398ad:::benchmark_data_smithi032_22800_object43196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.555583+0000", 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032287560000000002, 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035181129999999998, 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171310 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555583+0000", 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555583+0000", 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555584+0000", 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2599999999999998e-07 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555585+0000", 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6929999999999999e-06 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555589+0000", 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8160000000000004e-06 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.589 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555592+0000", 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.914e-06 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555629+0000", 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7208e-05 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555738+0000", 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010812899999999999 2024-04-18T22:24:17.590 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171312 5.4 5:2437e25a:::benchmark_data_smithi032_22800_object43198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.555742+0000", 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0030706980000000002, 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033941039999999998, 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171312 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555742+0000", 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555742+0000", 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555742+0000", 2024-04-18T22:24:17.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8800000000000002e-07 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555746+0000", 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4400000000000001e-06 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555753+0000", 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8050000000000007e-06 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555760+0000", 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5150000000000003e-06 2024-04-18T22:24:17.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557545+0000", 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017850450000000001 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557657+0000", 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011183699999999999 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171313 5.c 5:366d3409:::benchmark_data_smithi032_22800_object43199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.555909+0000", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0029032659999999998, 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032781199999999998, 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171313 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555909+0000", 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555909+0000", 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555909+0000", 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2900000000000004e-07 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555912+0000", 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9749999999999999e-06 2024-04-18T22:24:17.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555918+0000", 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9460000000000003e-06 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555922+0000", 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9079999999999997e-06 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557540+0000", 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016181539999999999 2024-04-18T22:24:17.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557642+0000", 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101755 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557722+0000", 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0190000000000003e-05 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.598 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171314 5.0 5:05b2ea7f:::benchmark_data_smithi032_22800_object43200:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.555963+0000", 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0028491039999999999, 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003289035, 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171314 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555963+0000", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555963+0000", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555964+0000", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.39e-07 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555965+0000", 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.796e-06 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555972+0000", 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7410000000000003e-06 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.555975+0000", 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2629999999999999e-06 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556019+0000", 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3433999999999998e-05 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556130+0000", 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110828 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556196+0000", 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6859000000000005e-05 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:171308 5.1 e55/49 5:8e6871d4:::benchmark_data_smithi032_22800_object43194:head v 55'10594, mlcod=55'10592)", 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.556030+0000", 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0027821899999999999, 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032600519999999998, 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556030+0000", 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556030+0000", 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556031+0000", 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0399999999999996e-07 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556035+0000", 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8489999999999997e-06 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556043+0000", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3710000000000002e-06 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556048+0000", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4979999999999997e-06 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556100+0000", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1375000000000002e-05 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556115+0000", 2024-04-18T22:24:17.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5452e-05 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:171315 5.4 5.37adb384 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.556501+0000", 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002311577, 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028187989999999999, 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:17.606 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 171315 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556501+0000", 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556501+0000", 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556502+0000", 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.85e-07 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556505+0000", 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9740000000000004e-06 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556516+0000", 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0644e-05 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.556522+0000", 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3450000000000002e-06 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:171311 5.1 e55/49 5:8011132b:::benchmark_data_smithi032_22800_object43197:head v 55'10595, mlcod=55'10592)", 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:17.557758+0000", 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010541509999999999, 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001589137, 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:17.609 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557758+0000", 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557758+0000", 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557759+0000", 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7800000000000001e-07 2024-04-18T22:24:17.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557763+0000", 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6519999999999997e-06 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557771+0000", 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9389999999999997e-06 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557784+0000", 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2471e-05 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557822+0000", 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8526999999999999e-05 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:17.557840+0000", 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7867e-05 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 14 2024-04-18T22:24:17.612 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:17.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:17.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:17.615 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:17.608+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:17.718 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:17.712+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:17.768 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:17.768 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:17.768 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:17.787 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:17.787 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:17.790 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:17.790 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:17.813 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:17.814 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:17.815 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:17.812+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:17.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:17.927 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:17.924+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:18.017 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:18.012+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:18.042 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:18.042 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:18.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:18.119 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:18.116+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:18.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:18.218+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:18.222 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:18.222 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:18.245 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:18.321 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:18.316+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:18.421 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:18.416+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:18.440 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:18.440 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:18.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:18.522 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:18.517+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:18.608 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:18.608 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:18.622 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:18.617+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:18.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:18.724 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:18.721+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:18.785 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:18.785 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:18.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:18.847 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:18.844+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:18.925 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:18.921+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:18.957 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:18.957 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:18.976 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:19.063 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:19.060+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:19.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:24:19.127 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:19.125+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:19.172 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:19.172 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:19.193 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:19.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:19.241+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:19.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:19.325+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:19.350 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:19.350 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:19.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:19.430 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:19.424+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:19.530 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:19.525+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:19.565 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:19.565 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:19.587 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:19.631 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:19.628+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:19.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:19.729+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:19.784 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:19.785 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:19.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:19.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:19.851 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:19.849+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:19.933 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:19.928+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:19.952 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:19.960 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177137 5.3 5.57ab7b33 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.960606+0000", 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.5148999999999998e-05, 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7108999999999997e-05, 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177137 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.961 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960606+0000", 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960606+0000", 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960606+0000", 2024-04-18T22:24:19.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1299999999999999e-07 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960610+0000", 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0679999999999998e-06 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960622+0000", 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1341e-05 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960626+0000", 2024-04-18T22:24:19.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5800000000000002e-06 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177121 5.4 5:2b67421f:::benchmark_data_smithi032_22800_object49007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.956907+0000", 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003734186, 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038357040000000001, 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177121 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956907+0000", 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956907+0000", 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956907+0000", 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8899999999999998e-07 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956912+0000", 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8400000000000002e-06 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956923+0000", 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.046e-05 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956931+0000", 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7970000000000001e-06 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956982+0000", 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1746999999999998e-05 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957100+0000", 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117318 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177122 5.0 5:0a26c4fd:::benchmark_data_smithi032_22800_object49008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.956959+0000", 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0036815160000000001, 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038466960000000001, 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177122 2024-04-18T22:24:19.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956959+0000", 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956959+0000", 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956960+0000", 2024-04-18T22:24:19.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0699999999999996e-07 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956966+0000", 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3820000000000001e-06 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956975+0000", 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7159999999999992e-06 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.956981+0000", 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1399999999999997e-06 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958326+0000", 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001344416 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958438+0000", 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112423 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958508+0000", 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9502e-05 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177124 5.3 5:c39c3ee7:::benchmark_data_smithi032_22800_object49010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.957062+0000", 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0035788539999999998, 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038103450000000001, 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177124 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957062+0000", 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957062+0000", 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957063+0000", 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4499999999999996e-07 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957067+0000", 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0840000000000002e-06 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957083+0000", 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5971999999999999e-05 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957090+0000", 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2899999999999997e-06 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957153+0000", 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3015999999999994e-05 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957301+0000", 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014826600000000001 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.957414+0000", 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011307 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177125 5.e 5:745a99ec:::benchmark_data_smithi032_22800_object49011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.958175+0000", 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0024661470000000001, 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002764346, 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:19.976 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177125 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958175+0000", 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958175+0000", 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958176+0000", 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8700000000000004e-07 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958180+0000", 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4710000000000001e-06 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958188+0000", 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6529999999999994e-06 2024-04-18T22:24:19.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958194+0000", 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6560000000000003e-06 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958483+0000", 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00028896299999999999 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958599+0000", 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115795 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177127 5.2 5:4381dbca:::benchmark_data_smithi032_22800_object49013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.958339+0000", 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023021909999999999, 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026601229999999999, 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177127 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958339+0000", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958339+0000", 2024-04-18T22:24:19.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958340+0000", 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5900000000000005e-07 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958343+0000", 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.766e-06 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958352+0000", 2024-04-18T22:24:19.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4249999999999998e-06 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958358+0000", 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7299999999999999e-06 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958698+0000", 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00033906700000000003 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958795+0000", 2024-04-18T22:24:19.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7923000000000004e-05 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958892+0000", 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6637000000000003e-05 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177128 5.0 5:00006344:::benchmark_data_smithi032_22800_object49014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.958372+0000", 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002268333, 2024-04-18T22:24:19.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026936569999999999, 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177128 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958372+0000", 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958372+0000", 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958373+0000", 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1200000000000003e-07 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958376+0000", 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7259999999999998e-06 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958411+0000", 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5333999999999997e-05 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958417+0000", 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2280000000000003e-06 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960024+0000", 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016068740000000001 2024-04-18T22:24:19.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960164+0000", 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013971400000000001 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960298+0000", 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013461899999999999 2024-04-18T22:24:19.988 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177132 5.a 5:5c8e45a5:::benchmark_data_smithi032_22800_object49018:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.958722+0000", 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001919221, 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024122549999999999, 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177132 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958722+0000", 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958722+0000", 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958722+0000", 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8899999999999999e-07 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958726+0000", 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7179999999999998e-06 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958734+0000", 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7829999999999999e-06 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958740+0000", 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0900000000000001e-06 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960117+0000", 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001376679 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960231+0000", 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011462400000000001 2024-04-18T22:24:19.992 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:177126 5.1 e55/49 5:8f905e19:::benchmark_data_smithi032_22800_object49012:head v 55'10954, mlcod=55'10952)", 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.958800+0000", 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00184125, 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023934170000000001, 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958800+0000", 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958800+0000", 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958800+0000", 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9299999999999998e-07 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958805+0000", 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9450000000000001e-06 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958813+0000", 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6510000000000001e-06 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958818+0000", 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7089999999999999e-06 2024-04-18T22:24:19.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958871+0000", 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.253e-05 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.958890+0000", 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9177999999999998e-05 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.996 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177134 5.a 5.c72719fa (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.959581+0000", 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010601040000000001, 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001663901, 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177134 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.959581+0000", 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.959581+0000", 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.959582+0000", 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9500000000000001e-07 2024-04-18T22:24:19.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.959585+0000", 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9079999999999997e-06 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.959595+0000", 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8859999999999999e-06 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.959602+0000", 2024-04-18T22:24:19.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4230000000000002e-06 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177135 5.3 5:c0ec572e:::benchmark_data_smithi032_22800_object49021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.960034+0000", 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000607261, 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001261009, 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177135 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960034+0000", 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960034+0000", 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960034+0000", 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0699999999999996e-07 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960037+0000", 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1350000000000001e-06 2024-04-18T22:24:20.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960045+0000", 2024-04-18T22:24:20.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0439999999999996e-06 2024-04-18T22:24:20.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960051+0000", 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6749999999999999e-06 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960108+0000", 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7083000000000002e-05 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960213+0000", 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104382 2024-04-18T22:24:20.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960556+0000", 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000343903 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:177136 5.a 5.4c23493a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.960460+0000", 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00018027200000000001, 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00089701699999999995, 2024-04-18T22:24:20.008 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 177136 2024-04-18T22:24:20.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.010 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960460+0000", 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960460+0000", 2024-04-18T22:24:20.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960461+0000", 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6599999999999996e-07 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960466+0000", 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9629999999999997e-06 2024-04-18T22:24:20.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960476+0000", 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5519999999999993e-06 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960483+0000", 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6899999999999992e-06 2024-04-18T22:24:20.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:177124 5.3 e55/19)", 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:19.960491+0000", 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00014974300000000001, 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00091231999999999999, 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960491+0000", 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960491+0000", 2024-04-18T22:24:20.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960492+0000", 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1299999999999999e-07 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960496+0000", 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.053e-06 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960504+0000", 2024-04-18T22:24:20.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9650000000000008e-06 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:19.960510+0000", 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1600000000000003e-06 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 13 2024-04-18T22:24:20.017 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:20.019 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:20.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:20.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:20.063 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:20.060+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:20.064 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:24:20.136 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:20.132+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:20.223 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:20.223 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:20.223 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:20.223 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:20.223 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:20.237 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.233+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:20.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.333+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.433+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.643646484000001, 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:20.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:20.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:20.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.451 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.641990105000005, 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.641865387999999, 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:20.457 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:20.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:20.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.640085729999996, 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:20.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:20.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:20.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.639603058999995, 2024-04-18T22:24:20.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:20.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:20.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:20.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.639363983999999, 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:20.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:20.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:20.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.639243213, 2024-04-18T22:24:20.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:20.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.636353800999998, 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.492 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.636231981999998, 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:20.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:20.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.636121532999994, 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:20.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.636052065000001, 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:20.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 74.635994924000002, 2024-04-18T22:24:20.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:20.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:20.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:20.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:20.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.406819413000001, 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.519 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:20.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:20.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:20.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:20.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:20.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.402176413999999, 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:20.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:20.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:20.536 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.393078676999998, 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.537 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:20.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:20.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.384058752000001, 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:20.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.550 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.368047384, 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:20.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:20.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:20.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 17.365243964000001, 2024-04-18T22:24:20.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167496 5.8 5:15a92cd7:::benchmark_data_smithi032_22800_object39382:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5737689540000002, 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073409221999999996, 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.569 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167496 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868626+0000", 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868629+0000", 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6619999999999998e-06 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868636+0000", 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.002e-06 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868642+0000", 2024-04-18T22:24:20.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5079999999999996e-06 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868675+0000", 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3651e-05 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868752+0000", 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6422999999999997e-05 2024-04-18T22:24:20.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941860+0000", 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073108464999999997 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941927+0000", 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6571999999999999e-05 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.942034+0000", 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010723 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167499 5.0 5:02d6841a:::benchmark_data_smithi032_22800_object39385:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:20.575 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5731434310000001, 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.074657837000000005, 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167499 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.576 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869261+0000", 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869264+0000", 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1719999999999999e-06 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869269+0000", 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8910000000000003e-06 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869274+0000", 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6959999999999996e-06 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:20.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870422+0000", 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00114858 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870532+0000", 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010966199999999999 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:20.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870646+0000", 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011456 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.937008+0000", 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.066361693999999999 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943734+0000", 2024-04-18T22:24:20.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067261580000000003 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943815+0000", 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1174000000000006e-05 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943908+0000", 2024-04-18T22:24:20.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2709999999999998e-05 2024-04-18T22:24:20.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.583 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:20.583 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:20.585 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.581+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:20.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.681+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178979 5.9 5:9dffcec2:::benchmark_data_smithi032_22800_object50865:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.740065+0000", 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0030109059999999998, 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003035603, 2024-04-18T22:24:20.743 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178979 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740065+0000", 2024-04-18T22:24:20.744 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740065+0000", 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740066+0000", 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2200000000000003e-07 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.745 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740070+0000", 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7309999999999999e-06 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740079+0000", 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.738e-06 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740085+0000", 2024-04-18T22:24:20.746 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0159999999999997e-06 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740204+0000", 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000119449 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740323+0000", 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011879699999999999 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.747 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178980 5.f 5:f87751b9:::benchmark_data_smithi032_22800_object50866:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.740597+0000", 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0024788829999999999, 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002576053, 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.748 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178980 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740597+0000", 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740597+0000", 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740598+0000", 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740603+0000", 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2899999999999996e-06 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740613+0000", 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0373e-05 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740630+0000", 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6883000000000001e-05 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741910+0000", 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001280263 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742020+0000", 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011001 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178981 5.1 5:869d5a7b:::benchmark_data_smithi032_22800_object50867:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.740647+0000", 2024-04-18T22:24:20.753 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002429488, 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025905020000000002, 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178981 2024-04-18T22:24:20.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740647+0000", 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740647+0000", 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740647+0000", 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6699999999999999e-07 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.756 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740650+0000", 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5639999999999999e-06 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740657+0000", 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7340000000000002e-06 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740661+0000", 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8040000000000001e-06 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741867+0000", 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001205766 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741980+0000", 2024-04-18T22:24:20.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112698 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742107+0000", 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012706999999999999 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178983 5.1 5.d86888d1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.741486+0000", 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0015900000000000001, 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018170339999999999, 2024-04-18T22:24:20.759 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178983 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741486+0000", 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741486+0000", 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741487+0000", 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1299999999999998e-07 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741493+0000", 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5149999999999997e-06 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741504+0000", 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0849999999999999e-05 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.741658+0000", 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015392800000000001 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.762 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178985 5.d 5:bb955a5e:::benchmark_data_smithi032_22800_object50871:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.742092+0000", 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00098449699999999998, 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012653320000000001, 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178985 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.763 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742092+0000", 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742092+0000", 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742092+0000", 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4899999999999995e-07 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742095+0000", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0620000000000001e-06 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742103+0000", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2779999999999996e-06 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742108+0000", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9509999999999997e-06 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742149+0000", 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1489000000000002e-05 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742230+0000", 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1005000000000001e-05 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.766 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:178984 5.2 e56/23 5:46a1f22d:::benchmark_data_smithi032_22800_object50870:head v 56'11239, mlcod=56'11235)", 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.742417+0000", 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000659484, 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001001184, 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742417+0000", 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742417+0000", 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742417+0000", 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7100000000000002e-07 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742422+0000", 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.826e-06 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742431+0000", 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0429999999999996e-06 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742437+0000", 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1179999999999997e-06 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742488+0000", 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0328000000000003e-05 2024-04-18T22:24:20.769 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742507+0000", 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8841000000000001e-05 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178986 5.9 5.f15bdfd9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.742507+0000", 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00056913299999999999, 2024-04-18T22:24:20.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00096398499999999999, 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178986 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742507+0000", 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742507+0000", 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742508+0000", 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1399999999999996e-07 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742512+0000", 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7859999999999998e-06 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.772 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742521+0000", 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1090000000000004e-06 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.742527+0000", 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1539999999999999e-06 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.773 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:178978 5.f 5:fae3d1bf:::benchmark_data_smithi032_22800_object50864:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.739761+0000", 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033156169999999999, 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003764206, 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 178978 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.774 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739761+0000", 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739761+0000", 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739762+0000", 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9400000000000004e-07 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739765+0000", 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5760000000000002e-06 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739776+0000", 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0454e-05 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739782+0000", 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.832e-06 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739844+0000", 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1404999999999999e-05 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739999+0000", 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000154649 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:178975 5.6 e56/19 5:6c4bd1c5:::benchmark_data_smithi032_22800_object50861:head v 56'11072, mlcod=56'11070)", 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:20.739927+0000", 2024-04-18T22:24:20.777 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0031496330000000002, 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036615490000000001, 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739927+0000", 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:20.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739927+0000", 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739927+0000", 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9799999999999999e-07 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739931+0000", 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0269999999999997e-06 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739938+0000", 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2420000000000003e-06 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.739944+0000", 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9000000000000003e-06 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740002+0000", 2024-04-18T22:24:20.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7985999999999997e-05 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:20.740019+0000", 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7050000000000001e-05 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:24:20.781 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:20.783 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:20.827 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.825+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.885+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.972 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:20.972 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:20.972 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:20.972 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:20.972 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:20.991 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:20.985+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:20.992 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:21.089 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:21.084+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:21.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~47s 2024-04-18T22:24:21.190 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:21.184+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:21.193 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:21.193 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:21.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:21.193 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:21.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.077215309, 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:21.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:21.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.198 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.077119463, 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:21.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.07706258100001, 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.208 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.077002909, 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:21.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:21.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:21.213 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.076944947, 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.214 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:21.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.076725255, 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:21.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:21.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:21.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.076437634, 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.224 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.07628516600001, 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.229 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:21.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:21.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:21.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.074685587, 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:21.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:21.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:21.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 109.074587235, 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:21.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:21.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.243 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 75.392865282000002, 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:21.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:21.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 75.392211068999998, 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:21.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:21.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 75.392104373999999, 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:21.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:21.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 75.390238464999996, 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.256 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:21.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:21.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 75.389573142000003, 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:21.260 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:21.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:21.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.264 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 7.8477574109999999, 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:21.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 7.8416169140000003, 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.270 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:21.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:21.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:21.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.275 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 5.3253108329999996, 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:21.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:21.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:21.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 5.3237769090000002, 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:21.281 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:21.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 5.3179515540000004, 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:21.287 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:21.292 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:21.294 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:21.289+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:21.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:21.394 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:21.388+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:21.451 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180670 5.7 5:e38da14d:::benchmark_data_smithi032_22800_object52556:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.448721+0000", 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023033630000000001, 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023293929999999999, 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180670 2024-04-18T22:24:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448721+0000", 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448721+0000", 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448721+0000", 2024-04-18T22:24:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.89e-07 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448724+0000", 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4439999999999998e-06 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448731+0000", 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1770000000000001e-06 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448737+0000", 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000003e-06 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448783+0000", 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5337999999999999e-05 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448904+0000", 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121317 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449005+0000", 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101291 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "MOSDPGPushReply(5.0 56/49 [PushReplyOp(5:08128b1e:::benchmark_data_smithi032_22800_object109359:head)])", 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.448720+0000", 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002304319, 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024297450000000001, 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:24:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448720+0000", 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448720+0000", 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448720+0000", 2024-04-18T22:24:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5900000000000003e-07 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448729+0000", 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0529999999999996e-06 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448738+0000", 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4400000000000005e-06 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448744+0000", 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7409999999999998e-06 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451085+0000", 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023412860000000001 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180673 5.3 5:ca5d3e95:::benchmark_data_smithi032_22800_object52559:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.448750+0000", 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022743339999999998, 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024594909999999998, 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180673 2024-04-18T22:24:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448750+0000", 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448750+0000", 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448750+0000", 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9900000000000001e-07 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448753+0000", 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.57e-06 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448760+0000", 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9059999999999996e-06 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448765+0000", 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1919999999999996e-06 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.450785+0000", 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002019907 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.450890+0000", 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105247 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180648 5.0 5:028ec311:::benchmark_data_smithi032_22800_object52534:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.438999+0000", 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.012025574000000001, 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012280127, 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180648 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.438999+0000", 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.438999+0000", 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.438999+0000", 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3599999999999999e-07 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439002+0000", 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.712e-06 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439008+0000", 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5309999999999998e-06 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439013+0000", 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8720000000000001e-06 2024-04-18T22:24:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439065+0000", 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2173999999999999e-05 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439167+0000", 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101164 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439249+0000", 2024-04-18T22:24:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2497000000000007e-05 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.442572+0000", 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033227500000000002 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180650 5.8 5:16ddfcf9:::benchmark_data_smithi032_22800_object52536:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.439156+0000", 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011868692, 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.012206986, 2024-04-18T22:24:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180650 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439156+0000", 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439156+0000", 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439156+0000", 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1299999999999999e-07 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439160+0000", 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0470000000000004e-06 2024-04-18T22:24:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439170+0000", 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4460000000000007e-06 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.439176+0000", 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4139999999999999e-06 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.442618+0000", 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034413859999999998 2024-04-18T22:24:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.442737+0000", 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118919 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180674 5.6 5:64d3e96e:::benchmark_data_smithi032_22800_object52560:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.448910+0000", 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021146170000000001, 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025224169999999999, 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180674 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448910+0000", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448910+0000", 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448910+0000", 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3499999999999996e-07 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448916+0000", 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1530000000000002e-06 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448924+0000", 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8669999999999994e-06 2024-04-18T22:24:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448930+0000", 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7969999999999997e-06 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448974+0000", 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4233999999999997e-05 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449087+0000", 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000112206 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449188+0000", 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101377 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180675 5.e 5:7e5e9838:::benchmark_data_smithi032_22800_object52561:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.449100+0000", 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001923828, 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024140099999999999, 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180675 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449100+0000", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449100+0000", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449101+0000", 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9100000000000003e-07 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449114+0000", 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2972999999999999e-05 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449122+0000", 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0169999999999996e-06 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449128+0000", 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2899999999999999e-06 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.450975+0000", 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018463430000000001 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451093+0000", 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117997 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180676 5.4 5:27e7960e:::benchmark_data_smithi032_22800_object52562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.449147+0000", 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0018776489999999999, 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024550980000000002, 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180676 2024-04-18T22:24:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449147+0000", 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449147+0000", 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449147+0000", 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5000000000000003e-07 2024-04-18T22:24:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449150+0000", 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8459999999999999e-06 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449159+0000", 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6349999999999995e-06 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449164+0000", 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2989999999999996e-06 2024-04-18T22:24:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449521+0000", 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00035719499999999998 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.449629+0000", 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107765 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180677 5.2 5:4e68c74e:::benchmark_data_smithi032_22800_object52563:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.451165+0000", 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998593, 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00052402799999999995, 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180677 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451165+0000", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451165+0000", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451166+0000", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5600000000000004e-07 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451171+0000", 2024-04-18T22:24:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.634e-06 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451180+0000", 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3049999999999998e-06 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451188+0000", 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0039999999999999e-06 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451239+0000", 2024-04-18T22:24:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1626999999999999e-05 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451344+0000", 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010431 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451451+0000", 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107459 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180678 5.4 5:2ab10512:::benchmark_data_smithi032_22800_object52564:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.451335+0000", 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9996891, 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00042807300000000001, 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180678 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451335+0000", 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451335+0000", 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451336+0000", 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4300000000000002e-07 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451340+0000", 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2969999999999997e-06 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451352+0000", 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2279000000000001e-05 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451360+0000", 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9719999999999992e-06 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451406+0000", 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5357e-05 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451517+0000", 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011100899999999999 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180679 5.e 5.1cb7e9be (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.451373+0000", 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9996514, 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00045528700000000002, 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180679 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451373+0000", 2024-04-18T22:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451373+0000", 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451373+0000", 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451376+0000", 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9210000000000001e-06 2024-04-18T22:24:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451384+0000", 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7330000000000003e-06 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451390+0000", 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0000000000000002e-06 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180661 5.8 5:1fa70cec:::benchmark_data_smithi032_22800_object52547:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.443109+0000", 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0079148789999999997, 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0087754189999999996, 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180661 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.443109+0000", 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.443109+0000", 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.443110+0000", 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.32e-07 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.443112+0000", 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.418e-06 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.443118+0000", 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8470000000000001e-06 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.443123+0000", 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9849999999999997e-06 2024-04-18T22:24:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.447749+0000", 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004625713 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.447863+0000", 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114647 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180665 5.8 5:1d9be102:::benchmark_data_smithi032_22800_object52551:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.446359+0000", 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0046649259999999998, 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0055962779999999997, 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180665 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446359+0000", 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446359+0000", 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446360+0000", 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3300000000000002e-07 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446366+0000", 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4110000000000001e-06 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446373+0000", 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.331e-06 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446377+0000", 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5950000000000001e-06 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451054+0000", 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046764939999999998 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.451168+0000", 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011385199999999999 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180667 5.4 5:240c159c:::benchmark_data_smithi032_22800_object52553:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.446673+0000", 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0043509430000000003, 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053608010000000001, 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180667 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446673+0000", 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446673+0000", 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446674+0000", 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4099999999999998e-07 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446678+0000", 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9400000000000004e-06 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446685+0000", 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.446689+0000", 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6619999999999996e-06 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448070+0000", 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001381183 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448187+0000", 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011643500000000001 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:180669 5.3 5:c9f799d7:::benchmark_data_smithi032_22800_object52555:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:21.448681+0000", 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002342865, 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003458633, 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 180669 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448681+0000", 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448681+0000", 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448682+0000", 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2600000000000002e-07 2024-04-18T22:24:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448686+0000", 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4630000000000001e-06 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448699+0000", 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3271999999999999e-05 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448706+0000", 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4410000000000004e-06 2024-04-18T22:24:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448752+0000", 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5895e-05 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448856+0000", 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010361699999999999 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:21.448957+0000", 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101313 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 15 2024-04-18T22:24:21.511 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:21.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:21.513 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:21.508+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:21.613 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:21.609+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:21.714 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:21.709+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:21.720 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:21.720 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:21.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:21.816 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:21.812+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.103166924000007, 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.101510544999996, 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.101385828000005, 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.099606170000001, 2024-04-18T22:24:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.099123499000001, 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.098884424000005, 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.098763653000006, 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.095874241000004, 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.095752422000004, 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.095641972999999, 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.095572505000007, 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 76.095515363999993, 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.866339852999999, 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.861696854000002, 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.852599117, 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.843579192, 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.827567823999999, 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 18.824764404, 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167496 5.8 5:15a92cd7:::benchmark_data_smithi032_22800_object39382:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 6.0332893939999996, 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073409221999999996, 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167496 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868626+0000", 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-04-18T22:24:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868629+0000", 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6619999999999998e-06 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868636+0000", 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.002e-06 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868642+0000", 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5079999999999996e-06 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868675+0000", 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3651e-05 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868752+0000", 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6422999999999997e-05 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941860+0000", 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073108464999999997 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941927+0000", 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6571999999999999e-05 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.942034+0000", 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010723 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167499 5.0 5:02d6841a:::benchmark_data_smithi032_22800_object39385:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 6.0326638709999996, 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.074657837000000005, 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167499 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869261+0000", 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869264+0000", 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1719999999999999e-06 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869269+0000", 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8910000000000003e-06 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869274+0000", 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6959999999999996e-06 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870422+0000", 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00114858 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870532+0000", 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010966199999999999 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870646+0000", 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011456 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.937008+0000", 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.066361693999999999 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943734+0000", 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067261580000000003 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943815+0000", 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1174000000000006e-05 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943908+0000", 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2709999999999998e-05 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:22.027 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:22.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:22.029 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:22.025+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:22.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:22.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:24:22.130 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:22.124+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:182597 5.3 e56/19 5:c65cbc26:::benchmark_data_smithi032_22800_object54483:head v 56'11736, mlcod=56'11733)", 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.194124+0000", 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0031267270000000002, 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00314394, 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.197 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194124+0000", 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194124+0000", 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194124+0000", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7199999999999998e-07 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194131+0000", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.331e-06 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194142+0000", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.102e-05 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194153+0000", 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0927999999999999e-05 2024-04-18T22:24:22.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194201+0000", 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8196999999999999e-05 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194222+0000", 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0812e-05 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.200 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:182607 5.5 5:aa98a5d8:::benchmark_data_smithi032_22800_object54493:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.194785+0000", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0024649049999999999, 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025216040000000002, 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 182607 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194785+0000", 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194785+0000", 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194786+0000", 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.75e-07 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194790+0000", 2024-04-18T22:24:22.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1710000000000004e-06 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194803+0000", 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.259e-05 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194808+0000", 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5110000000000003e-06 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194878+0000", 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9726000000000003e-05 2024-04-18T22:24:22.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195056+0000", 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000178424 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195184+0000", 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012789999999999999 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:182603 5.0 e56/49 5:0f5688f4:::benchmark_data_smithi032_22800_object54489:head v 56'11379, mlcod=56'11377)", 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.194958+0000", 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0022922770000000001, 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00239127, 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194958+0000", 2024-04-18T22:24:22.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194958+0000", 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194958+0000", 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2300000000000002e-07 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194962+0000", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.844e-06 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194971+0000", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2560000000000002e-06 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.194976+0000", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.412e-06 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195103+0000", 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012691800000000001 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195115+0000", 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2119e-05 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:182608 5.3 e56/19 5:cf8a288b:::benchmark_data_smithi032_22800_object54494:head v 56'11737, mlcod=56'11733)", 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.195861+0000", 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001389136, 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001518978, 2024-04-18T22:24:22.208 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195861+0000", 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195861+0000", 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195862+0000", 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5199999999999996e-07 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195867+0000", 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9740000000000001e-06 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195877+0000", 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7489999999999993e-06 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195885+0000", 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1529999999999997e-06 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195944+0000", 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8836999999999999e-05 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.195964+0000", 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9902e-05 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.211 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:182605 5.c e56/19 5:377bd681:::benchmark_data_smithi032_22800_object54491:head v 56'11406, mlcod=56'11404)", 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.196539+0000", 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00071112200000000001, 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00089413800000000005, 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196539+0000", 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196539+0000", 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196540+0000", 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.44e-07 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196545+0000", 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0640000000000001e-06 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196554+0000", 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8929999999999994e-06 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196560+0000", 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3010000000000003e-06 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196633+0000", 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3394999999999998e-05 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.196654+0000", 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1050999999999999e-05 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:182589 5.6 e56/19 5:69bad4c6:::benchmark_data_smithi032_22800_object54475:head v 56'11319, mlcod=56'11315)", 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.197045+0000", 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00020549199999999999, 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00044912499999999998, 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197045+0000", 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197045+0000", 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197046+0000", 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4199999999999995e-07 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197050+0000", 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.972e-06 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197061+0000", 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0638999999999999e-05 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197066+0000", 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1909999999999999e-06 2024-04-18T22:24:22.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197112+0000", 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5657e-05 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197135+0000", 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3215999999999999e-05 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:182609 5.f 5.589ad09f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.197453+0000", 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9997978, 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106665, 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 182609 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.219 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197453+0000", 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197453+0000", 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197454+0000", 2024-04-18T22:24:22.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1930000000000001e-06 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197459+0000", 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9420000000000003e-06 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197471+0000", 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2418e-05 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197478+0000", 2024-04-18T22:24:22.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.306e-06 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197543+0000", 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4801000000000005e-05 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:182610 5.9 5.2dccbc39 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:22.197530+0000", 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9997201, 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4079000000000005e-05, 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:22.222 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 182610 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197530+0000", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197530+0000", 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:22.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197531+0000", 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5499999999999997e-07 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197535+0000", 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8589999999999998e-06 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197544+0000", 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6589999999999996e-06 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:22.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:22.197550+0000", 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1630000000000001e-06 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 8 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:22.225 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:22.227 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:22.228 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:22.228 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:22.230 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:22.231 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:22.231 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:22.228+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:22.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:22.329+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:22.381 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:22.381 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:22.401 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:22.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:22.441+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:22.533 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:22.528+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:22.594 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:22.594 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:22.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:22.634 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:22.629+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:22.735 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:22.728+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:22.769 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:22.770 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:22.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:22.835 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:22.833+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:22.935 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:22.932+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:22.983 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:22.983 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:23.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:23.036 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:23.033+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:23.137 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:23.132+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:23.196 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:23.196 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:23.214 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:23.232 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:24:23.259 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:23.257+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:23.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:23.337+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:23.367 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:23.367 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:23.389 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:23.475 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:23.473+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:23.541 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:23.537+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:23.585 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:23.585 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:23.605 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:23.642 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:23.636+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:23.743 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:23.741+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:23.801 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:23.801 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:23.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:23.867 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:23.864+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:23.944 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:23.940+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:23.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:24.045 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:24.041+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:24.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:24.141+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:24.189 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:24.189 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:24.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:24.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:24.255 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:24.252+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:24.348 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:24.344+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:24.361 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:24.361 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:24.380 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:24.398 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:24.418 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:24.422 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:24.422 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:24.448 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:24.444+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:188174 5.5 5:adeb1d6e:::benchmark_data_smithi032_22800_object60060:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:24.533623+0000", 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0043364440000000001, 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0043633170000000002, 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:24.537 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 188174 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533623+0000", 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533623+0000", 2024-04-18T22:24:24.538 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533624+0000", 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3510000000000001e-06 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533630+0000", 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9569999999999999e-06 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533643+0000", 2024-04-18T22:24:24.539 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3190000000000001e-05 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533660+0000", 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6980999999999999e-05 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533761+0000", 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100519 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.540 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.533888+0000", 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012703599999999999 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.534079+0000", 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00019093800000000001 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536082+0000", 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020033429999999999 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:188177 5.7 e57/23 5:ecff7085:::benchmark_data_smithi032_22800_object60063:head v 57'11815, mlcod=57'11811)", 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:24.535808+0000", 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0021508069999999998, 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022585919999999998, 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535808+0000", 2024-04-18T22:24:24.542 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535808+0000", 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535809+0000", 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2699999999999999e-07 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535813+0000", 2024-04-18T22:24:24.543 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4689999999999999e-06 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535824+0000", 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0953e-05 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535832+0000", 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.622e-06 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.544 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535877+0000", 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4604000000000002e-05 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.535897+0000", 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0678000000000001e-05 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.538061+0000", 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021637370000000002 2024-04-18T22:24:24.545 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:188178 5.1 5:8ad4355b:::benchmark_data_smithi032_22800_object60064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:24.536506+0000", 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0014528169999999999, 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00163116, 2024-04-18T22:24:24.546 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.547 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.547 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:24.547 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:24.547 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:24.547 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:24.547 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 188178 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536506+0000", 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.548 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.549 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.549 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.549 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536506+0000", 2024-04-18T22:24:24.549 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536507+0000", 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.063e-06 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.550 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536513+0000", 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8599999999999998e-06 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536526+0000", 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2312e-05 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536534+0000", 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0530000000000007e-06 2024-04-18T22:24:24.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536590+0000", 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6561000000000003e-05 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536709+0000", 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011830800000000001 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:24.536775+0000", 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6251e-05 2024-04-18T22:24:24.552 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.553 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.553 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.553 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.553 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:24.553 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 3 2024-04-18T22:24:24.553 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:24.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:24.549+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:24.558 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:24.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:24.653+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:24.756 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:24.753+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:24.756 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:24.756 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:24.756 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:24.756 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:24.757 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:24.776 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:24.856 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:24.852+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:24.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:24.953+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:24.974 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.858253644, 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:24.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:24.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.85815779799999, 2024-04-18T22:24:24.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:24.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.858100916, 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:24.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.85804124400001, 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:24.990 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:24.995 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.85798328200001, 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:24.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.000 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.85776359, 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.85747596900001, 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.006 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:25.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.857323501, 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:25.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:25.016 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.855723922, 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:25.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:25.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:25.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:25.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 112.85562557, 2024-04-18T22:24:25.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:25.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:25.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 79.173903616999993, 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.026 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:25.030 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.033 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.033 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.033 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.033 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 79.173249404000003, 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.034 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.035 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.036 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.037 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 79.173142709000004, 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.038 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:25.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:25.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:25.041 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 79.171276800000001, 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.042 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:25.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.044 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.045 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:25.046 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 79.170611476999994, 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:25.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:25.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:25.050 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 11.628795746, 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:25.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:25.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 11.622655248999999, 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:25.056 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:25.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.062 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 9.1063491679999995, 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:25.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:25.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:25.067 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 9.1048152439999992, 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.068 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:25.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:25.073 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 9.0989898890000003, 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:25.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:25.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:25.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.079 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:25.081 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:25.077+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:25.081 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:25.181 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:25.177+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189850 5.8 5:19ac28ee:::benchmark_data_smithi032_22800_object61736:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.228026+0000", 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0042283360000000001, 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042472600000000001, 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189850 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228026+0000", 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228026+0000", 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228027+0000", 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7400000000000003e-07 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228031+0000", 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.63e-06 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228040+0000", 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2609999999999999e-06 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228048+0000", 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9030000000000004e-06 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.229941+0000", 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00189314 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230070+0000", 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012905600000000001 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189852 5.8 5:11746aa6:::benchmark_data_smithi032_22800_object61738:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.230709+0000", 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0015457240000000001, 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016070349999999999, 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189852 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230709+0000", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230709+0000", 2024-04-18T22:24:25.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230710+0000", 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0299999999999998e-07 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230714+0000", 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2019999999999997e-06 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230724+0000", 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7259999999999997e-06 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230729+0000", 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5729999999999998e-06 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.231888+0000", 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011588830000000001 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.232025+0000", 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013704200000000001 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189833 5.7 5:e8ea0e98:::benchmark_data_smithi032_22800_object61719:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.221366+0000", 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.010888761, 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010986862, 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189833 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221366+0000", 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221366+0000", 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221366+0000", 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7800000000000001e-07 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221370+0000", 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.202e-06 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221376+0000", 2024-04-18T22:24:25.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.46e-06 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221383+0000", 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8530000000000003e-06 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221419+0000", 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6275000000000001e-05 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221503+0000", 2024-04-18T22:24:25.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4237e-05 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221609+0000", 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105812 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.226626+0000", 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050167010000000001 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189834 5.7 5:ec33c8a8:::benchmark_data_smithi032_22800_object61720:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.221395+0000", 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.010859377999999999, 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010997962, 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189834 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221395+0000", 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221395+0000", 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221396+0000", 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7800000000000002e-07 2024-04-18T22:24:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221398+0000", 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4779999999999998e-06 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221405+0000", 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6730000000000003e-06 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221410+0000", 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.339e-06 2024-04-18T22:24:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.223439+0000", 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020293680000000001 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.223547+0000", 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010739300000000001 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.223658+0000", 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111221 2024-04-18T22:24:25.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228754+0000", 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050958660000000001 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189837 5.6 5:601cb6bf:::benchmark_data_smithi032_22800_object61723:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.221479+0000", 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.010775824, 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.010957214, 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189837 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221479+0000", 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221479+0000", 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221479+0000", 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9299999999999998e-07 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221482+0000", 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6790000000000001e-06 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221491+0000", 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6209999999999993e-06 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.221496+0000", 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.485e-06 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.223384+0000", 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001887875 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.223508+0000", 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012406200000000001 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.223621+0000", 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011296800000000001 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228950+0000", 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053293799999999999 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189842 5.7 5:ede4cef5:::benchmark_data_smithi032_22800_object61728:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.224181+0000", 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0080731599999999994, 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0082945699999999994, 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189842 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224181+0000", 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224181+0000", 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224182+0000", 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2399999999999997e-07 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224185+0000", 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0970000000000002e-06 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224196+0000", 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0777e-05 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224203+0000", 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5200000000000003e-06 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.225402+0000", 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011989500000000001 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.225543+0000", 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000141051 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.225604+0000", 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1290000000000004e-05 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.230868+0000", 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052641249999999997 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189844 5.0 5:0d3d27fa:::benchmark_data_smithi032_22800_object61730:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.224518+0000", 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0077362029999999997, 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0079994899999999997, 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189844 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224518+0000", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224518+0000", 2024-04-18T22:24:25.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224519+0000", 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7899999999999997e-07 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224524+0000", 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3309999999999997e-06 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224533+0000", 2024-04-18T22:24:25.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9600000000000006e-06 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224541+0000", 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1159999999999999e-06 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224596+0000", 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5435e-05 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224786+0000", 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00018949799999999999 2024-04-18T22:24:25.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224915+0000", 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012896000000000001 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.231864+0000", 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0069493039999999999 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189845 5.c 5:3f9b0590:::benchmark_data_smithi032_22800_object61731:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.224714+0000", 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0075406600000000002, 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0078439630000000007, 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189845 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224714+0000", 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224714+0000", 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224715+0000", 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7899999999999998e-07 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224718+0000", 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.817e-06 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224727+0000", 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1729999999999995e-06 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.224733+0000", 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9819999999999996e-06 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.226062+0000", 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013296499999999999 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.226195+0000", 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000133077 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.226313+0000", 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117358 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.231198+0000", 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004885108 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189847 5.7 5:e40a9880:::benchmark_data_smithi032_22800_object61733:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.227201+0000", 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0050531700000000001, 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053955490000000004, 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.267 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189847 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227201+0000", 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227201+0000", 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227202+0000", 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1399999999999996e-07 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227207+0000", 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4710000000000001e-06 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227216+0000", 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4760000000000004e-06 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227221+0000", 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3029999999999999e-06 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227266+0000", 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4719999999999999e-05 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227360+0000", 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3832000000000006e-05 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227421+0000", 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1327000000000004e-05 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.231217+0000", 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003796008 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:189849 5.c 5:36cb2e88:::benchmark_data_smithi032_22800_object61735:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:25.227263+0000", 2024-04-18T22:24:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0049918150000000001, 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053735110000000001, 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 189849 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227263+0000", 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227263+0000", 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227263+0000", 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5400000000000002e-07 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227265+0000", 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3180000000000001e-06 2024-04-18T22:24:25.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227271+0000", 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8610000000000003e-06 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.227276+0000", 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7169999999999999e-06 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228291+0000", 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010148189999999999 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228427+0000", 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013584 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.228549+0000", 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122333 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:25.231244+0000", 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026954370000000002 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.275 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.276 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:25.276 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 10 2024-04-18T22:24:25.276 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:25.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:24:25.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:25.323 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:25.320+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:25.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:25.381+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:25.425 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:25.426 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:25.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:25.484 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:25.481+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:25.585 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:25.580+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:25.642 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:25.642 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:25.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.841759929000006, 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:25.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:25.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:25.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.840103549999995, 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.839978833000004, 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.838199175, 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:25.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:25.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:25.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:25.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.837716503999999, 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:25.665 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:25.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:25.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.837477429000003, 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:25.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:25.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.837356658000004, 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:25.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.834467246000003, 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:25.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.834345427000002, 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:25.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.834234977999998, 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.693 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.698 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.834165510000005, 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 79.834108369000006, 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.704 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:25.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.709 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.604932858000002, 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.713 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.600289859, 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.721 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.591192121999999, 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.722 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:25.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:25.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:25.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.582172196999998, 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:25.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.729 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:25.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:25.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.566160829000001, 2024-04-18T22:24:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:25.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:25.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:25.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 22.563357408999998, 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:25.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:25.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167496 5.8 5:15a92cd7:::benchmark_data_smithi032_22800_object39382:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.7718823990000008, 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073409221999999996, 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.747 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167496 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868626+0000", 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868629+0000", 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6619999999999998e-06 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868636+0000", 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.002e-06 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868642+0000", 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5079999999999996e-06 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868675+0000", 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3651e-05 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868752+0000", 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6422999999999997e-05 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941860+0000", 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073108464999999997 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941927+0000", 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6571999999999999e-05 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.942034+0000", 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010723 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167499 5.0 5:02d6841a:::benchmark_data_smithi032_22800_object39385:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 9.7712568760000007, 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.074657837000000005, 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167499 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:25.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869261+0000", 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869264+0000", 2024-04-18T22:24:25.753 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1719999999999999e-06 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869269+0000", 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8910000000000003e-06 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869274+0000", 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6959999999999996e-06 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870422+0000", 2024-04-18T22:24:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00114858 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870532+0000", 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010966199999999999 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870646+0000", 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011456 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.937008+0000", 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.066361693999999999 2024-04-18T22:24:25.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943734+0000", 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067261580000000003 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943815+0000", 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1174000000000006e-05 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943908+0000", 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2709999999999998e-05 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.756 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.757 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:25.757 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:25.757 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:25.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:25.759 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:25.753+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:25.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:25.853+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:25.918 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:25.918 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:25.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.918 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:191498 5.5 5:a2915cbb:::benchmark_data_smithi032_22800_object63384:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.912203+0000", 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0067133779999999999, 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067308960000000001, 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 191498 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912203+0000", 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912203+0000", 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912204+0000", 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3399999999999998e-07 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912208+0000", 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5209999999999998e-06 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912217+0000", 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6630000000000005e-06 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912224+0000", 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7390000000000002e-06 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912274+0000", 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9715000000000001e-05 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912361+0000", 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.6723000000000003e-05 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.912426+0000", 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5821999999999994e-05 2024-04-18T22:24:25.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.915679+0000", 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032528119999999999 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:191502 5.1 5:8ca679d3:::benchmark_data_smithi032_22800_object63388:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.913430+0000", 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0054870070000000003, 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0055581670000000001, 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.923 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 191502 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913430+0000", 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913430+0000", 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913431+0000", 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1850000000000001e-06 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913435+0000", 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8279999999999996e-06 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913445+0000", 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0043e-05 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913453+0000", 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2339999999999994e-06 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913529+0000", 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5462000000000004e-05 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913666+0000", 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013720599999999999 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.913785+0000", 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011898 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918623+0000", 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0048385440000000002 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918958+0000", 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033509499999999999 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:191506 5.d 5:b6c6dd36:::benchmark_data_smithi032_22800_object63392:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.916185+0000", 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0027314409999999998, 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0028522510000000001, 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.928 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 191506 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916185+0000", 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916185+0000", 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916187+0000", 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2100000000000001e-06 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916191+0000", 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0430000000000001e-06 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916203+0000", 2024-04-18T22:24:25.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1867e-05 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916209+0000", 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9619999999999996e-06 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916257+0000", 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7252000000000001e-05 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916378+0000", 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000121526 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:191508 5.1 5:85c69a52:::benchmark_data_smithi032_22800_object63394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.916250+0000", 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0026667850000000001, 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0028247609999999999, 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.932 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 191508 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916250+0000", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916250+0000", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916251+0000", 2024-04-18T22:24:25.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9599999999999999e-07 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916254+0000", 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.0369999999999999e-06 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916266+0000", 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2568999999999999e-05 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916273+0000", 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.392e-06 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916319+0000", 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5974999999999999e-05 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916448+0000", 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012959399999999999 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916563+0000", 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011482800000000001 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918847+0000", 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022843939999999999 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:191511 5.5 5:ae5d95e1:::benchmark_data_smithi032_22800_object63397:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-18T22:24:25.936 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.916284+0000", 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0026329159999999999, 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0028353430000000002, 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 191511 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916284+0000", 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916284+0000", 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916284+0000", 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7199999999999999e-07 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916287+0000", 2024-04-18T22:24:25.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3700000000000002e-06 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916294+0000", 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3660000000000003e-06 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916299+0000", 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5469999999999998e-06 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916353+0000", 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4755e-05 2024-04-18T22:24:25.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916504+0000", 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150292 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916643+0000", 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000139276 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.940 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:191507 5.c e57/19 5:34bc3de9:::benchmark_data_smithi032_22800_object63393:head v 57'11947, mlcod=57'11945)", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.916560+0000", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0023571339999999999, 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0025981440000000001, 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916560+0000", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916560+0000", 2024-04-18T22:24:25.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916560+0000", 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7300000000000001e-07 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916567+0000", 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0929999999999997e-06 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916575+0000", 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0250000000000002e-06 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916582+0000", 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9199999999999998e-06 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916633+0000", 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0361000000000001e-05 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916652+0000", 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9828999999999999e-05 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:191510 5.2 e57/23 5:4d78bcbb:::benchmark_data_smithi032_22800_object63396:head v 57'12004, mlcod=57'12002)", 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.916626+0000", 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002291314, 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002562922, 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.944 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916626+0000", 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916626+0000", 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916626+0000", 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9599999999999999e-07 2024-04-18T22:24:25.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916630+0000", 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9310000000000001e-06 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916637+0000", 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9750000000000001e-06 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916643+0000", 2024-04-18T22:24:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3430000000000001e-06 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916690+0000", 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6446999999999998e-05 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916709+0000", 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9587999999999999e-05 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:191512 5.6 e57/19 5:6af8ff0f:::benchmark_data_smithi032_22800_object63398:head v 57'11879, mlcod=57'11877)", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.916891+0000", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002026014, 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002332102, 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916891+0000", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916891+0000", 2024-04-18T22:24:25.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916891+0000", 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0299999999999999e-07 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916898+0000", 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3040000000000001e-06 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916907+0000", 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3060000000000003e-06 2024-04-18T22:24:25.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916912+0000", 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2129999999999999e-06 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916965+0000", 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2819999999999999e-05 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.916985+0000", 2024-04-18T22:24:25.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.001e-05 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "MOSDPGPush(5.0 57/49 [PushOp(5:0b04d934:::benchmark_data_smithi032_22800_object76454:head, version: 35'4811, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5:0b04d934:::benchmark_data_smithi032_22800_object76454:head@35'4811, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.918674+0000", 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00024266700000000001, 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00059431100000000004, 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918674+0000", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918674+0000", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918675+0000", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9100000000000003e-07 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918722+0000", 2024-04-18T22:24:25.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7414999999999999e-05 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918745+0000", 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2765999999999999e-05 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918763+0000", 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7419e-05 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918897+0000", 2024-04-18T22:24:25.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013455800000000001 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918911+0000", 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4171999999999999e-05 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "pg_backfill(progress 5.0 e 57/57 lb 5:0b039253:::benchmark_data_smithi032_22800_object74207:head)", 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.918794+0000", 2024-04-18T22:24:25.954 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000123168, 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000506307, 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918794+0000", 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918794+0000", 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918794+0000", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.5199999999999998e-07 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918803+0000", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9039999999999998e-06 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918811+0000", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4910000000000005e-06 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918821+0000", 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7149999999999993e-06 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop_reply(client.4586.0:191508 5.1 e57/49 ondisk, result = 0)", 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.918914+0000", 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 2.6120000000000001e-06, 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040924100000000002, 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:25.957 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918914+0000", 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918914+0000", 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918915+0000", 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.8200000000000001e-07 2024-04-18T22:24:25.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918917+0000", 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.565e-06 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918923+0000", 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5979999999999996e-06 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918927+0000", 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1269999999999996e-06 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.919161+0000", 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00023427300000000001 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.919169+0000", 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3399999999999998e-06 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.919323+0000", 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015405599999999999 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.960 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:191509 5.c e57/19)", 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:25.918933+0000", 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.9999843, 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00045956600000000001, 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918933+0000", 2024-04-18T22:24:25.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918933+0000", 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918933+0000", 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2700000000000001e-07 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918937+0000", 2024-04-18T22:24:25.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6239999999999999e-06 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918942+0000", 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3519999999999997e-06 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:25.918946+0000", 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0589999999999996e-06 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:25.963 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 12 2024-04-18T22:24:25.964 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:25.965 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:25.965 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:25.960+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.065 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.060+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.163 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:26.163 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:26.164 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:26.164 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:26.164 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:26.166 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.160+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:26.251 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:24:26.267 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.264+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.23654575, 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.236449904, 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.360 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:26.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:26.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:26.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.236393022, 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:26.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:26.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:26.370 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.23633335, 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.371 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:26.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:26.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:26.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:26.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.236275388, 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:26.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.23605569599999, 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.235768075, 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.386 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:26.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:26.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.235615607, 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:26.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.395 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.234016028, 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:26.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:26.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:26.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 114.233917676, 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.402 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.404 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 80.552195722999997, 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:26.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:26.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:26.408 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 80.551541510000007, 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.409 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.410 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:26.411 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.412 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 80.551434814999993, 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:26.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 80.549568906000005, 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.417 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.418 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:26.419 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.420 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 80.548903582999998, 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.421 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.422 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:26.424 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 13.007087852, 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:26.425 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.426 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:26.427 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:26.428 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:26.429 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.430 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 13.000947354999999, 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.431 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:26.432 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:26.433 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:26.434 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 10.484641273999999, 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.436 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.437 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:26.438 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:26.440 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.441 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 10.483107349999999, 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.442 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.443 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.444 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.445 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.446 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 10.477281995, 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:26.447 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:26.448 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:26.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:26.450 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:26.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:26.452 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:26.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:26.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:26.456 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.452+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.554 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:26.549+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:26.611 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:26.611 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:26.611 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:26.612 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:26.612 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:26.629 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:26.629 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:26.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:26.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:26.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:26.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:26.653+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:26.756 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.752+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.784 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:26.784 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:26.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:26.896 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.892+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.956 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:26.952+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:26.995 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:26.995 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:27.017 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:27.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~53s 2024-04-18T22:24:27.063 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:27.061+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:27.157 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.152+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.170 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:27.170 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:27.190 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:27.258 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.252+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:27.357+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:27.386 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:27.386 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:27.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:27.459 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.456+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.560 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.556+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.605 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:27.605 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:27.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:27.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:27.657+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:27.761 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.756+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.771 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:27.772 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:27.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:27.879 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.876+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.963 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:27.960+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:27.990 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:27.990 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:28.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:28.095 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:28.092+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:28.164 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:28.160+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:28.203 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:28.203 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:28.223 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:28.267 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:28.265+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:28.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:24:28.366 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:28.360+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:28.377 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:28.377 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:28.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:28.485 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:28.481+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:28.568 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:28.564+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:28.591 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:28.591 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:28.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:28.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:24:28.669 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:28.664+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:28.770 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:28.765+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:28.784 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:28.785 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:28.802 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:28.805 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:28.806 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.6918922, 2024-04-18T22:24:28.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:28.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:28.813 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.691796354, 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:28.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.69173947199999, 2024-04-18T22:24:28.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:28.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.6916798, 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.691621838, 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:28.830 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.835 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.691402146, 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:28.840 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.691114525, 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.69096205699999, 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.846 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:28.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.68936247800001, 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 116.689264126, 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.007542173000004, 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.860 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.00688796, 2024-04-18T22:24:28.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:28.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:28.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.868 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.006781265000001, 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:28.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.004915355999998, 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:28.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:28.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:28.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 83.004250033000005, 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 15.462434302, 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:28.881 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:28.886 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 15.456293805, 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:28.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:28.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:28.891 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 12.939987724, 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.892 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:28.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:28.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 12.9384538, 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:28.897 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:28.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:28.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 12.932628445000001, 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:28.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:28.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:28.908 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:28.909 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:28.910 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:28.904+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:29.010 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:29.004+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:198952 5.2 e59/23 ondisk, result = 0)", 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066130+0000", 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00012720900000000001, 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014499599999999999, 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:29.068 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066130+0000", 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066130+0000", 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066131+0000", 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5300000000000004e-07 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.069 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066133+0000", 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.509e-06 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066143+0000", 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6809999999999993e-06 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066148+0000", 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8720000000000001e-06 2024-04-18T22:24:29.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066194+0000", 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5667999999999997e-05 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066203+0000", 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1940000000000004e-06 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.071 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198962 5.3 5:c1186223:::benchmark_data_smithi032_22800_object70848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066242+0000", 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 1.5173e-05, 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010204099999999999, 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198962 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.072 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066242+0000", 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066242+0000", 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066243+0000", 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9999999999999998e-07 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066246+0000", 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7640000000000001e-06 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066252+0000", 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9499999999999998e-06 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066255+0000", 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.737e-06 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066288+0000", 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3040999999999998e-05 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198964 5.0 5:02498d1a:::benchmark_data_smithi032_22800_object70850:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066284+0000", 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999738, 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109079, 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:24:29.075 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198964 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066284+0000", 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066284+0000", 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066285+0000", 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9699999999999995e-07 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066287+0000", 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3910000000000001e-06 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066292+0000", 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9180000000000002e-06 2024-04-18T22:24:29.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066296+0000", 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.676e-06 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066327+0000", 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1210000000000001e-05 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.078 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198965 5.b 5.d540b14b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066406+0000", 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998512, 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6814000000000001e-05, 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198965 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.079 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066406+0000", 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066406+0000", 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066407+0000", 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7999999999999995e-07 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.080 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066409+0000", 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.576e-06 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066416+0000", 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.562e-06 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066419+0000", 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.298e-06 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198966 5.4 5.7c915ac4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066430+0000", 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998274, 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2765e-05, 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198966 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.082 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066430+0000", 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066430+0000", 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066430+0000", 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2500000000000001e-07 2024-04-18T22:24:29.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066432+0000", 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0320000000000002e-06 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066437+0000", 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8840000000000002e-06 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066442+0000", 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4050000000000002e-06 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.084 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066488+0000", 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6570999999999998e-05 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198967 5.a 5:5a4988d0:::benchmark_data_smithi032_22800_object70853:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066454+0000", 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998031, 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2719000000000005e-05, 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198967 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066454+0000", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066454+0000", 2024-04-18T22:24:29.086 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066455+0000", 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5400000000000002e-07 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066456+0000", 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7519999999999999e-06 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066461+0000", 2024-04-18T22:24:29.087 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0010000000000001e-06 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066465+0000", 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.444e-06 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066511+0000", 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5751999999999998e-05 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198969 5.3 5.1c0fcd13 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066474+0000", 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.999783, 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013278199999999999, 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198969 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066474+0000", 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066474+0000", 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066475+0000", 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2599999999999998e-07 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.090 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066477+0000", 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.897e-06 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066481+0000", 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9359999999999998e-06 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066486+0000", 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3499999999999999e-06 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198970 5.8 5:10534162:::benchmark_data_smithi032_22800_object70856:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.066500+0000", 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997578, 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016676500000000001, 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198970 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.092 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066500+0000", 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066500+0000", 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066500+0000", 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:29.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066503+0000", 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5210000000000001e-06 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066509+0000", 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.725e-06 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066514+0000", 2024-04-18T22:24:29.094 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9450000000000001e-06 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.066553+0000", 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8637000000000003e-05 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198953 5.e 5:7ec5b303:::benchmark_data_smithi032_22800_object70839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.061236+0000", 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0050216820000000004, 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054960199999999999, 2024-04-18T22:24:29.095 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198953 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061236+0000", 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061236+0000", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061237+0000", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2500000000000004e-07 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061241+0000", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3139999999999997e-06 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061251+0000", 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7599999999999997e-06 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061257+0000", 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6059999999999999e-06 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061349+0000", 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1817999999999994e-05 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061421+0000", 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2157999999999996e-05 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198954 5.b 5:d3871745:::benchmark_data_smithi032_22800_object70840:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.061276+0000", 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.004981236, 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005521064, 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:29.099 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198954 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061276+0000", 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061276+0000", 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061277+0000", 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1699999999999998e-07 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061280+0000", 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.785e-06 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061286+0000", 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8109999999999997e-06 2024-04-18T22:24:29.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061292+0000", 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2959999999999998e-06 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061342+0000", 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0022999999999999e-05 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061453+0000", 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110924 2024-04-18T22:24:29.102 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198957 5.8 5:174ebd72:::benchmark_data_smithi032_22800_object70843:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.061711+0000", 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0045462920000000004, 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051246449999999997, 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198957 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061711+0000", 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061711+0000", 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061712+0000", 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4099999999999998e-07 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061715+0000", 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3699999999999999e-06 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061724+0000", 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8580000000000006e-06 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061729+0000", 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6999999999999999e-06 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061770+0000", 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1114e-05 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.061881+0000", 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111285 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198958 5.6 5:628b9667:::benchmark_data_smithi032_22800_object70844:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.063461+0000", 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.002796406, 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034095369999999998, 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198958 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.107 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063461+0000", 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063461+0000", 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063462+0000", 2024-04-18T22:24:29.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2900000000000003e-07 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063465+0000", 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3129999999999999e-06 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063474+0000", 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1579999999999994e-06 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063481+0000", 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8870000000000003e-06 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063533+0000", 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2104000000000001e-05 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063655+0000", 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012156599999999999 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063765+0000", 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110109 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198959 5.e 5:70319c90:::benchmark_data_smithi032_22800_object70845:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.063499+0000", 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0027584860000000001, 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034081110000000001, 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:29.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198959 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063499+0000", 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063499+0000", 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063500+0000", 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3000000000000001e-07 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063502+0000", 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6690000000000002e-06 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063515+0000", 2024-04-18T22:24:29.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2696e-05 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063521+0000", 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8320000000000002e-06 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063553+0000", 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2713999999999999e-05 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063647+0000", 2024-04-18T22:24:29.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3332999999999996e-05 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:198960 5.3 5:c4b95b1c:::benchmark_data_smithi032_22800_object70846:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:29.063607+0000", 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026507810000000001, 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033484000000000001, 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 198960 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063607+0000", 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063607+0000", 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063607+0000", 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6899999999999997e-07 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063611+0000", 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3799999999999998e-06 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063620+0000", 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9050000000000003e-06 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063626+0000", 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9150000000000001e-06 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063673+0000", 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7574000000000002e-05 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063782+0000", 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010870399999999999 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:29.063892+0000", 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011039099999999999 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 14 2024-04-18T22:24:29.119 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:29.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:29.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:29.117+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:29.221 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:29.217+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:29.317 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:29.317 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:29.317 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:29.317 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:29.317 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:29.324 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:29.320+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:29.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:29.423 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:29.420+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:29.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:29.521+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:29.530 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.729287826999993, 2024-04-18T22:24:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:29.538 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.727631447999997, 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.539 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:29.544 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.727506731000005, 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:29.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:29.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:29.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:29.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.725727073000002, 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:29.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:29.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.725244402000001, 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.725005327000005, 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.561 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:29.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.566 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.724884556000006, 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.721995144000005, 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:29.572 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:29.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.721873325000004, 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.578 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:29.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:29.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:29.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:29.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.721762876, 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:29.583 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:29.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:29.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:29.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.721693407999993, 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:29.594 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 83.721636266999994, 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:29.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:29.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.492460756, 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:29.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:29.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:29.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.487817756999998, 2024-04-18T22:24:29.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:29.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:29.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.478720020000001, 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:29.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:29.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:29.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.469700095, 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.619 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:29.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:29.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.453688726999999, 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:29.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:29.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 26.450885307, 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:29.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:29.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:29.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:29.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167496 5.8 5:15a92cd7:::benchmark_data_smithi032_22800_object39382:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.659410297000001, 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073409221999999996, 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167496 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868626+0000", 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868629+0000", 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6619999999999998e-06 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868636+0000", 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.002e-06 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868642+0000", 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5079999999999996e-06 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868675+0000", 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3651e-05 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868752+0000", 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6422999999999997e-05 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941860+0000", 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073108464999999997 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941927+0000", 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6571999999999999e-05 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.942034+0000", 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010723 2024-04-18T22:24:29.642 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167499 5.0 5:02d6841a:::benchmark_data_smithi032_22800_object39385:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 13.658784774000001, 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.074657837000000005, 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167499 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869261+0000", 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869264+0000", 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1719999999999999e-06 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869269+0000", 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8910000000000003e-06 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869274+0000", 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6959999999999996e-06 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870422+0000", 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00114858 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870532+0000", 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010966199999999999 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:29.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870646+0000", 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011456 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.937008+0000", 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.066361693999999999 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943734+0000", 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067261580000000003 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:29.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943815+0000", 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1174000000000006e-05 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943908+0000", 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2709999999999998e-05 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:29.648 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:29.650 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:29.650 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:29.644+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:29.750 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:29.748+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:29.806 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:29.806 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:29.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:200681 5.d 5:ba542538:::benchmark_data_smithi032_22800_object72567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.806621+0000", 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000125318, 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014984199999999999, 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "reached pg", 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 200681 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806621+0000", 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806621+0000", 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806621+0000", 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9999999999999997e-07 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806625+0000", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7290000000000002e-06 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806634+0000", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0610000000000002e-06 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806640+0000", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6470000000000003e-06 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806682+0000", 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1904999999999999e-05 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:200666 5.5 5:a35a156d:::benchmark_data_smithi032_22800_object72552:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.801175+0000", 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0055709080000000003, 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.005673928, 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:29.810 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 200666 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801175+0000", 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801175+0000", 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801176+0000", 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.9000000000000005e-07 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801181+0000", 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4859999999999999e-06 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801201+0000", 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0384e-05 2024-04-18T22:24:29.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801219+0000", 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7900000000000001e-05 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801273+0000", 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4129000000000003e-05 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801395+0000", 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012132 2024-04-18T22:24:29.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.801462+0000", 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7218000000000006e-05 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806099+0000", 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046369410000000003 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.814 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:200667 5.7 e59/23 5:e6595d02:::benchmark_data_smithi032_22800_object72553:head v 59'12634, mlcod=59'12631)", 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.802958+0000", 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003788067, 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00397328, 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.802958+0000", 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.802958+0000", 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.802959+0000", 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7000000000000004e-07 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.802964+0000", 2024-04-18T22:24:29.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2820000000000004e-06 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.802974+0000", 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4960000000000002e-06 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.802981+0000", 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.917e-06 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.804151+0000", 2024-04-18T22:24:29.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001170152 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.804168+0000", 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7401000000000002e-05 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:200672 5.5 5:a7c25a68:::benchmark_data_smithi032_22800_object72558:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.803361+0000", 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033855389999999999, 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00363423, 2024-04-18T22:24:29.818 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 200672 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803361+0000", 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803361+0000", 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803361+0000", 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8400000000000004e-07 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803365+0000", 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.4359999999999998e-06 2024-04-18T22:24:29.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803374+0000", 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3300000000000005e-06 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803381+0000", 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6039999999999998e-06 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803427+0000", 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5911000000000001e-05 2024-04-18T22:24:29.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803558+0000", 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013124799999999999 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.803701+0000", 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014311400000000001 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806168+0000", 2024-04-18T22:24:29.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0024674969999999999 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:200674 5.7 e59/23 5:e1fc491f:::benchmark_data_smithi032_22800_object72560:head v 59'12635, mlcod=59'12631)", 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.806126+0000", 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00062021600000000004, 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000954035, 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806126+0000", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806126+0000", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806127+0000", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1699999999999997e-07 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806133+0000", 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2249999999999997e-06 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806144+0000", 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1161e-05 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806152+0000", 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9410000000000007e-06 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806216+0000", 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4187000000000001e-05 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806235+0000", 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8842e-05 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:200676 5.f 5:f1e591fe:::benchmark_data_smithi032_22800_object72562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.806219+0000", 2024-04-18T22:24:29.826 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00052760299999999997, 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00092405999999999999, 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 200676 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806219+0000", 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806219+0000", 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806220+0000", 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0800000000000003e-07 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806223+0000", 2024-04-18T22:24:29.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.749e-06 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806233+0000", 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2539999999999998e-06 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806239+0000", 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5969999999999997e-06 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806360+0000", 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000121366 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806489+0000", 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012804599999999999 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:200677 5.5 5:a2594238:::benchmark_data_smithi032_22800_object72563:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.806250+0000", 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00049607299999999998, 2024-04-18T22:24:29.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00096236499999999997, 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 200677 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806250+0000", 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806250+0000", 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806251+0000", 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6600000000000002e-07 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806253+0000", 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4719999999999998e-06 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806260+0000", 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5740000000000001e-06 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806264+0000", 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7169999999999999e-06 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806358+0000", 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3572000000000005e-05 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806467+0000", 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010872 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806560+0000", 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3318000000000003e-05 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:200680 5.9 5:9e5da193:::benchmark_data_smithi032_22800_object72566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:29.834 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:29.806573+0000", 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00017332599999999999, 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00069203899999999998, 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 200680 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806573+0000", 2024-04-18T22:24:29.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806573+0000", 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806574+0000", 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1500000000000003e-07 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806578+0000", 2024-04-18T22:24:29.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3800000000000004e-06 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806589+0000", 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0691e-05 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806595+0000", 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5549999999999999e-06 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806654+0000", 2024-04-18T22:24:29.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8393000000000001e-05 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:29.806801+0000", 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014714200000000001 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 8 2024-04-18T22:24:29.838 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:29.839 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:29.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:29.881+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:29.952 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:29.949+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:29.989 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:29.989 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:29.990 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:29.990 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:29.990 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:30.009 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:30.095 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:30.093+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:30.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:30.149+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:30.204 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:30.204 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:30.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:30.204 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.087751502, 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:30.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:30.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:30.210 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.087655656, 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:30.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:30.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.087598774, 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.221 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.08753910199999, 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:30.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:30.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:30.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.227 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.08748113999999, 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:30.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:30.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.232 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.08726144800001, 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.08697382699999, 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.238 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:30.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.243 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.086821359, 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:30.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:30.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:30.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.08522178, 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:30.249 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:30.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:30.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 118.085123428, 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:30.258 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 84.403401474999995, 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:30.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:30.261 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.262 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 84.402747262000005, 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.263 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.264 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:30.265 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.266 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:30.267 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:30.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.267 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.267 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.268 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 84.402640567000006, 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.269 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:30.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:30.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:30.272 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 84.400774658000003, 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.273 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.274 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:30.275 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.276 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 84.400109334999996, 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:30.277 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.278 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.279 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.280 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:30.281 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 16.858293604, 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:30.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:30.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:30.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 16.852153106999999, 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.288 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.289 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.289 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:30.289 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:30.289 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.289 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:30.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.290 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:30.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:30.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.293 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:30.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:30.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:30.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:30.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.335847026, 2024-04-18T22:24:30.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.334313101999999, 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.304 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:30.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:30.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:30.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 14.328487747, 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:30.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:30.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:30.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:30.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:30.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:30.315 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:30.317 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:30.313+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:30.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:30.417 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:30.413+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:30.471 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202219 5.6 5:62eccafa:::benchmark_data_smithi032_22800_object74105:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.471 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.463858+0000", 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0063625419999999997, 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063904210000000003, 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202219 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463858+0000", 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463858+0000", 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463859+0000", 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6000000000000003e-07 2024-04-18T22:24:30.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463863+0000", 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3610000000000003e-06 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463873+0000", 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5699999999999999e-06 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463879+0000", 2024-04-18T22:24:30.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5599999999999999e-06 2024-04-18T22:24:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466236+0000", 2024-04-18T22:24:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023571590000000002 2024-04-18T22:24:30.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466374+0000", 2024-04-18T22:24:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013764800000000001 2024-04-18T22:24:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466507+0000", 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013323999999999999 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202221 5.e 5:703ddf73:::benchmark_data_smithi032_22800_object74107:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.463907+0000", 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0063137669999999996, 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064261630000000004, 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.480 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202221 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463907+0000", 2024-04-18T22:24:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463907+0000", 2024-04-18T22:24:30.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463907+0000", 2024-04-18T22:24:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6300000000000005e-07 2024-04-18T22:24:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463910+0000", 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7099999999999999e-06 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463918+0000", 2024-04-18T22:24:30.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2090000000000005e-06 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.463923+0000", 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1909999999999999e-06 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466567+0000", 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002643304 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466686+0000", 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000119521 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202225 5.8 5:1e245265:::benchmark_data_smithi032_22800_object74111:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.465002+0000", 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0052183749999999999, 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054078310000000001, 2024-04-18T22:24:30.486 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202225 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465002+0000", 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465002+0000", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465003+0000", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2699999999999998e-07 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465008+0000", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9699999999999998e-06 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465017+0000", 2024-04-18T22:24:30.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9630000000000004e-06 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465024+0000", 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8499999999999996e-06 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465762+0000", 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00073795000000000002 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465885+0000", 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012281 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202227 5.7 5:e491fe22:::benchmark_data_smithi032_22800_object74113:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.465910+0000", 2024-04-18T22:24:30.490 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0043109159999999997, 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045842119999999998, 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202227 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465910+0000", 2024-04-18T22:24:30.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465910+0000", 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465910+0000", 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3399999999999998e-07 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465914+0000", 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1480000000000002e-06 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465920+0000", 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8530000000000003e-06 2024-04-18T22:24:30.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465925+0000", 2024-04-18T22:24:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.425e-06 2024-04-18T22:24:30.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.465961+0000", 2024-04-18T22:24:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5831999999999998e-05 2024-04-18T22:24:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466033+0000", 2024-04-18T22:24:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2760000000000001e-05 2024-04-18T22:24:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.466093+0000", 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9802000000000001e-05 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202229 5.c 5:346bed30:::benchmark_data_smithi032_22800_object74115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.467175+0000", 2024-04-18T22:24:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003045903, 2024-04-18T22:24:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033991289999999999, 2024-04-18T22:24:30.498 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202229 2024-04-18T22:24:30.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467175+0000", 2024-04-18T22:24:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467175+0000", 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467175+0000", 2024-04-18T22:24:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5099999999999999e-07 2024-04-18T22:24:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467180+0000", 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5589999999999999e-06 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467191+0000", 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0555e-05 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467197+0000", 2024-04-18T22:24:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6760000000000001e-06 2024-04-18T22:24:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467278+0000", 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0853999999999995e-05 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467390+0000", 2024-04-18T22:24:30.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011213300000000001 2024-04-18T22:24:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467492+0000", 2024-04-18T22:24:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101439 2024-04-18T22:24:30.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202230 5.b 5:d1942ec7:::benchmark_data_smithi032_22800_object74116:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.507 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.467223+0000", 2024-04-18T22:24:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0029973270000000002, 2024-04-18T22:24:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034273260000000001, 2024-04-18T22:24:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202230 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467223+0000", 2024-04-18T22:24:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467223+0000", 2024-04-18T22:24:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467224+0000", 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467226+0000", 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7259999999999998e-06 2024-04-18T22:24:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467235+0000", 2024-04-18T22:24:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2679999999999994e-06 2024-04-18T22:24:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467240+0000", 2024-04-18T22:24:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.135e-06 2024-04-18T22:24:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468636+0000", 2024-04-18T22:24:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013963929999999999 2024-04-18T22:24:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468742+0000", 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106092 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.516 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:202224 5.5 e59/49 5:afadaf6b:::benchmark_data_smithi032_22800_object74110:head v 59'12512, mlcod=59'12506)", 2024-04-18T22:24:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.467978+0000", 2024-04-18T22:24:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022422140000000002, 2024-04-18T22:24:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027380270000000001, 2024-04-18T22:24:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467978+0000", 2024-04-18T22:24:30.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467978+0000", 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467979+0000", 2024-04-18T22:24:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3000000000000001e-07 2024-04-18T22:24:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467984+0000", 2024-04-18T22:24:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8119999999999998e-06 2024-04-18T22:24:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.467994+0000", 2024-04-18T22:24:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.927e-06 2024-04-18T22:24:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468001+0000", 2024-04-18T22:24:30.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1359999999999999e-06 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468046+0000", 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5096000000000003e-05 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468097+0000", 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0708999999999998e-05 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202231 5.4 5:246c9481:::benchmark_data_smithi032_22800_object74117:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.468381+0000", 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001839652, 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024015389999999998, 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202231 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468381+0000", 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468381+0000", 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468381+0000", 2024-04-18T22:24:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3499999999999996e-07 2024-04-18T22:24:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468384+0000", 2024-04-18T22:24:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.01e-06 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468393+0000", 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2060000000000007e-06 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468398+0000", 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8069999999999996e-06 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468445+0000", 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6751e-05 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468581+0000", 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013536700000000001 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.531 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202232 5.b 5.88d1528b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.468671+0000", 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001549358, 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002176678, 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202232 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468671+0000", 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468671+0000", 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468672+0000", 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5899999999999995e-07 2024-04-18T22:24:30.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468676+0000", 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7069999999999998e-06 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468684+0000", 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3760000000000008e-06 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468690+0000", 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1079999999999998e-06 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202233 5.a 5:57551e25:::benchmark_data_smithi032_22800_object74119:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.468703+0000", 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001517282, 2024-04-18T22:24:30.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002200862, 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202233 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468703+0000", 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468703+0000", 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468704+0000", 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7899999999999999e-07 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468706+0000", 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.491e-06 2024-04-18T22:24:30.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468712+0000", 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.733e-06 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468716+0000", 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5870000000000003e-06 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468761+0000", 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4771000000000002e-05 2024-04-18T22:24:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469014+0000", 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00025323300000000001 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202234 5.6 5:6999f3f6:::benchmark_data_smithi032_22800_object74120:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.468728+0000", 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001492197, 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022691289999999999, 2024-04-18T22:24:30.539 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202234 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468728+0000", 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468728+0000", 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468729+0000", 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3700000000000001e-07 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468731+0000", 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9420000000000002e-06 2024-04-18T22:24:30.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468737+0000", 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.477e-06 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468743+0000", 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9270000000000002e-06 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468860+0000", 2024-04-18T22:24:30.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117402 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469124+0000", 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026320599999999999 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469224+0000", 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100466 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202235 5.a 5:57ff43a8:::benchmark_data_smithi032_22800_object74121:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.468948+0000", 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0012723439999999999, 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021391600000000002, 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202235 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468948+0000", 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468948+0000", 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468949+0000", 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.68e-07 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468952+0000", 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.331e-06 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468959+0000", 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0890000000000002e-06 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468964+0000", 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.305e-06 2024-04-18T22:24:30.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470625+0000", 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016605509999999999 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470738+0000", 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113202 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202236 5.2 5:49bfd7f2:::benchmark_data_smithi032_22800_object74122:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.468980+0000", 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001240667, 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002190759, 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202236 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468980+0000", 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468980+0000", 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468980+0000", 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8500000000000002e-07 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468983+0000", 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3690000000000001e-06 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468988+0000", 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5960000000000003e-06 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.468993+0000", 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6800000000000001e-06 2024-04-18T22:24:30.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470618+0000", 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016246819999999999 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470740+0000", 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259200000000001 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470852+0000", 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111347 2024-04-18T22:24:30.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202237 5.e 5:73a1e608:::benchmark_data_smithi032_22800_object74123:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.469001+0000", 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0012196290000000001, 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022483400000000001, 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202237 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469001+0000", 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469001+0000", 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469001+0000", 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6300000000000001e-07 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469003+0000", 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.911e-06 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469009+0000", 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3430000000000003e-06 2024-04-18T22:24:30.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469013+0000", 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.143e-06 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470191+0000", 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0011782299999999999 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470312+0000", 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012157 2024-04-18T22:24:30.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202238 5.2 5.5dbb4ac2 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.469107+0000", 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00111371, 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0022078810000000001, 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202238 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469107+0000", 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469107+0000", 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469107+0000", 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4499999999999997e-07 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469111+0000", 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5049999999999999e-06 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469119+0000", 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9130000000000003e-06 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469125+0000", 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0669999999999997e-06 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:202239 5.4 5:2587c1ae:::benchmark_data_smithi032_22800_object74125:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.469182+0000", 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001038237, 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021880960000000001, 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 202239 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469182+0000", 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469182+0000", 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469183+0000", 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1900000000000002e-07 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469186+0000", 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3560000000000001e-06 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469194+0000", 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0909999999999993e-06 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.469200+0000", 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9309999999999996e-06 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470559+0000", 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001358827 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.470673+0000", 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000113583 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:202229 5.c e59/19)", 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.471121+0000", 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9990993, 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031797900000000003, 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.471121+0000", 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.471121+0000", 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.471122+0000", 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6199999999999998e-07 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.471125+0000", 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9490000000000001e-06 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.471132+0000", 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7249999999999997e-06 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.471138+0000", 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8699999999999997e-06 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.565 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:30.566 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 17 2024-04-18T22:24:30.566 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:30.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:30.568 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:30.565+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:30.668 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:30.664+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:30.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:24:30.721 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:30.721 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:30.722 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:30.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:30.783 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:30.780+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:30.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:30.870 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:30.864+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25655 5.1 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:30.930 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.129574594000005, 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092456195000000005, 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25655 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798747+0000", 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798748+0000", 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0699999999999996e-07 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798871+0000", 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012284700000000001 2024-04-18T22:24:30.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798881+0000", 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9729999999999992e-06 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.798888+0000", 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.985e-06 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.799976+0000", 2024-04-18T22:24:30.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010884639999999999 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800082+0000", 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010584199999999999 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800166+0000", 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4013999999999999e-05 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802504+0000", 2024-04-18T22:24:30.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002337595 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891027+0000", 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088523227999999995 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891130+0000", 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103016 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.891203+0000", 2024-04-18T22:24:30.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3423999999999994e-05 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25656 5.c 5:32839d89:::benchmark_data_smithi032_22800_object25655:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.127918214999994, 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091914132999999995, 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25656 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800404+0000", 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9500000000000002e-07 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800494+0000", 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9192999999999998e-05 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800503+0000", 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8949999999999993e-06 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800511+0000", 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1040000000000001e-06 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800563+0000", 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2204999999999999e-05 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800699+0000", 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013598000000000001 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800835+0000", 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013671000000000001 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804290+0000", 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0034542879999999998 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892135+0000", 2024-04-18T22:24:30.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087845690000000004 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892216+0000", 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0833000000000003e-05 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892318+0000", 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010153999999999999 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25657 5.6 5:617f461b:::benchmark_data_smithi032_22800_object25656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.127793498000003, 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091803387, 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25657 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.942 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800528+0000", 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800529+0000", 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7599999999999997e-07 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800602+0000", 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2596999999999996e-05 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800610+0000", 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2519999999999999e-06 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800616+0000", 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2539999999999998e-06 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800660+0000", 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4335999999999997e-05 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800799+0000", 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000138634 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.800935+0000", 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013646099999999999 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.804191+0000", 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032552689999999999 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892182+0000", 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.087991312000000002 2024-04-18T22:24:30.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892268+0000", 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6153e-05 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892332+0000", 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3542999999999998e-05 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.947 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25660 5.5 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.126013839999999, 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090191207999999995, 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25660 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802308+0000", 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802309+0000", 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5000000000000001e-07 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802412+0000", 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103434 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802421+0000", 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.337e-06 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802427+0000", 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.913e-06 2024-04-18T22:24:30.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802476+0000", 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9245e-05 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802607+0000", 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013144900000000001 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802800+0000", 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019310700000000001 2024-04-18T22:24:30.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805517+0000", 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027165240000000001 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892331+0000", 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814525000000003 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892426+0000", 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4446999999999999e-05 2024-04-18T22:24:30.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.892499+0000", 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3376999999999999e-05 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25662 5.3 5:c7ce96d6:::benchmark_data_smithi032_22800_object25661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.125531168999998, 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093087207000000005, 2024-04-18T22:24:30.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25662 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802791+0000", 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1600000000000001e-07 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802829+0000", 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7543999999999998e-05 2024-04-18T22:24:30.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802836+0000", 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6909999999999997e-06 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802843+0000", 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.336e-06 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.802890+0000", 2024-04-18T22:24:30.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7172000000000002e-05 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803016+0000", 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012607499999999999 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803155+0000", 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013888099999999999 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805490+0000", 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023353639999999999 2024-04-18T22:24:30.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895721+0000", 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090231073999999994 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895814+0000", 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2594000000000001e-05 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895878+0000", 2024-04-18T22:24:30.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3960000000000004e-05 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25663 5.c 5:37d92baa:::benchmark_data_smithi032_22800_object25662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.125292094000002, 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090678515000000001, 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.959 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25663 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803030+0000", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803106+0000", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6199999999999995e-05 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803131+0000", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4295e-05 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803135+0000", 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0099999999999997e-06 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803190+0000", 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4911999999999999e-05 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803284+0000", 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4748999999999997e-05 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803371+0000", 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6086000000000002e-05 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805462+0000", 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002091318 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893515+0000", 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088053192000000002 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893615+0000", 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100389 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893708+0000", 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2843999999999994e-05 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25665 5.2 5:4a98c1a6:::benchmark_data_smithi032_22800_object25664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.964 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.125171323000004, 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090781973000000002, 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25665 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803151+0000", 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0200000000000002e-07 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803199+0000", 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8198999999999997e-05 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803207+0000", 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3509999999999996e-06 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803214+0000", 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6310000000000003e-06 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803263+0000", 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8344999999999999e-05 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803375+0000", 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011203499999999999 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.803497+0000", 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012230800000000001 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.805531+0000", 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020342379999999998 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893604+0000", 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088073235999999999 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893782+0000", 2024-04-18T22:24:30.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000177491 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.893933+0000", 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015063699999999999 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25667 5.3 5:c4c62574:::benchmark_data_smithi032_22800_object25666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:30.970 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.122281911000002, 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090193018999999999, 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25667 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:30.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806040+0000", 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806041+0000", 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.029e-06 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806098+0000", 2024-04-18T22:24:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6821999999999999e-05 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806126+0000", 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7772000000000001e-05 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806134+0000", 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806195+0000", 2024-04-18T22:24:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0643999999999997e-05 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806338+0000", 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014372399999999999 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806491+0000", 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015306200000000001 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809135+0000", 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026435690000000001 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895939+0000", 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086804214000000005 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896116+0000", 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017715100000000001 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.896233+0000", 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116641 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25668 5.3 5:cd1e668f:::benchmark_data_smithi032_22800_object25667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.122160092000001, 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092670844000000002, 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25668 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806162+0000", 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806226+0000", 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4045000000000001e-05 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806236+0000", 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7710000000000001e-06 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806241+0000", 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9119999999999997e-06 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807542+0000", 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001300453 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807631+0000", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9724999999999999e-05 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807715+0000", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4104999999999995e-05 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.809541+0000", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825595 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898610+0000", 2024-04-18T22:24:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089068888999999998 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898719+0000", 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109182 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898833+0000", 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0001136 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25669 5.3 5:c9d5b318:::benchmark_data_smithi032_22800_object25668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.122049642999997, 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.092967626999999997, 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25669 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806272+0000", 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806273+0000", 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3399999999999999e-07 2024-04-18T22:24:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806313+0000", 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0120000000000002e-05 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806319+0000", 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2310000000000001e-06 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806323+0000", 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.391e-06 2024-04-18T22:24:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808358+0000", 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020341030000000002 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808447+0000", 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9394000000000005e-05 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808536+0000", 2024-04-18T22:24:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9364999999999996e-05 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.810362+0000", 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001825562 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.898966+0000", 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088604077000000003 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.986 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899123+0000", 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000157501 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.899240+0000", 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011634899999999999 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.987 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25670 5.1 5:83ec06bf:::benchmark_data_smithi032_22800_object25669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.121980175000004, 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.089465295, 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25670 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806342+0000", 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5200000000000002e-07 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806372+0000", 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0131999999999999e-05 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806383+0000", 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0266e-05 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806387+0000", 2024-04-18T22:24:30.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.792e-06 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806428+0000", 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0812999999999997e-05 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806509+0000", 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0383000000000006e-05 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1231999999999997e-05 2024-04-18T22:24:30.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.808655+0000", 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020649829999999998 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895640+0000", 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086985001000000006 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895715+0000", 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4827999999999996e-05 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.895807+0000", 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2412999999999998e-05 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:25671 5.2 5:4eef9d87:::benchmark_data_smithi032_22800_object25670:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 85.121923034000005, 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088477820999999998, 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 25671 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:30.994 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806399+0000", 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1500000000000005e-07 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806461+0000", 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1373999999999999e-05 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:30.995 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806468+0000", 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5000000000000002e-06 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806473+0000", 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4229999999999998e-06 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806516+0000", 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3550000000000001e-05 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.996 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806590+0000", 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4016000000000004e-05 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.806674+0000", 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3382999999999998e-05 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.807818+0000", 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001144583 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.997 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894633+0000", 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.086814381999999995 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894803+0000", 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000170054 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:23:05.894877+0000", 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4041000000000005e-05 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.998 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.892747523000001, 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:31.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:31.002 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:31.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.004 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.888104523999999, 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:31.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:31.007 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:31.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:31.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.879006787000002, 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:31.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:31.012 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:31.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:31.014 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:31.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:31.016 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.869986862000001, 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.017 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:31.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:31.019 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.020 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.021 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.853975494, 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:31.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:31.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:31.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:31.029 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 27.851172074000001, 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:31.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:31.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:31.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:31.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167496 5.8 5:15a92cd7:::benchmark_data_smithi032_22800_object39382:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 15.059697064, 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073409221999999996, 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.036 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167496 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868625+0000", 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868626+0000", 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0900000000000001e-07 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868629+0000", 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6619999999999998e-06 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868636+0000", 2024-04-18T22:24:31.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.002e-06 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868642+0000", 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5079999999999996e-06 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868675+0000", 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3651e-05 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.868752+0000", 2024-04-18T22:24:31.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6422999999999997e-05 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941860+0000", 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.073108464999999997 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.941927+0000", 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6571999999999999e-05 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.942034+0000", 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010723 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:167499 5.0 5:02d6841a:::benchmark_data_smithi032_22800_object39385:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 15.059071541, 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.074657837000000005, 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:31.041 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 167499 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869250+0000", 2024-04-18T22:24:31.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869261+0000", 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869264+0000", 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1719999999999999e-06 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869269+0000", 2024-04-18T22:24:31.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8910000000000003e-06 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.869274+0000", 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6959999999999996e-06 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870422+0000", 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00114858 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870532+0000", 2024-04-18T22:24:31.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010966199999999999 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.870646+0000", 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011456 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.937008+0000", 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.066361693999999999 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:31.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943734+0000", 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067261580000000003 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943815+0000", 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1174000000000006e-05 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:15.943908+0000", 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2709999999999998e-05 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.046 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:31.047 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:31.047 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:31.048 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:31.049 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:31.049 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:31.049 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:31.050 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:31.050 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:31.051 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:31.051 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:31.045+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:31.149 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:31.145+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:31.211 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:31.211 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:31.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:31.250 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:31.244+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:31.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:31.345+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:31.431 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:31.432 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:31.451 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:31.448+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:31.452 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:31.552 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:31.549+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:31.647 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:31.647 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:31.654 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:31.648+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:31.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:24:31.754 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:31.749+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:31.820 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:31.820 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:31.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:31.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:31.853+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:31.956 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:31.952+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:31.990 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:31.990 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:32.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:32.095 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:32.092+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:32.158 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:32.153+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:32.202 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:32.202 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:32.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:32.271 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:32.269+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:32.360 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:32.356+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:32.382 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:32.382 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:32.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:32.495 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:32.493+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:32.562 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:32.557+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:32.589 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:32.589 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:32.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:32.662 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:32.656+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:32.763 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:32.760+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:32.790 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:32.790 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:32.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:32.863 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~59s 2024-04-18T22:24:32.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:32.861+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:32.927 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:32.927 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:32.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:32.987 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:32.984+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:33.046 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:33.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:33.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:33.066 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.061+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.167 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.165+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.215 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:33.215 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:33.216 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.181811378999999, 2024-04-18T22:24:33.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:33.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:33.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:33.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.177168380000001, 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.227 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:33.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.168070643, 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:33.233 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:33.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.159050718, 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.240 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:33.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:33.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.245 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.143039349999999, 2024-04-18T22:24:33.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:33.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:33.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 30.140235929999999, 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:33.252 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:33.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3307369659999999, 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.259 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:33.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:33.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.264 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.32907076, 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3280982639999999, 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:33.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.273 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.326738287, 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:33.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:33.281 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3241711569999999, 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:33.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:33.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.284 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:33.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.323116518, 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.292 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.322274578, 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.295 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.298 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.320461468, 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:33.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.303 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3202744169999998, 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:33.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:33.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:33.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3200356960000001, 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:33.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:33.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3198464909999998, 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:33.319 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:33.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:33.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3197349539999998, 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:33.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.328 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.329 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.3195581289999998, 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.332 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.334 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 2.317382404, 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:33.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:33.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:33.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:33.339 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:33.341 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:33.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:33.342 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:33.342 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:33.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.341+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.437+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.471 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:33.472 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:33.472 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 0 2024-04-18T22:24:33.472 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:33.485 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:33.571 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.569+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.643 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.641+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.649 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:33.649 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:33.649 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:33.649 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:33.649 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:33.663 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:33.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:24:33.744 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:33.740+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:33.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:33.841+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:33.857 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.741205024, 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.858 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:33.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.741109178, 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:33.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.74105229600001, 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.869 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.740992624, 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:33.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:33.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:33.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.740934662, 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.880 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:33.885 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.74071497, 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:33.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:33.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.740427349, 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.891 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:33.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.740274881, 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:33.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:33.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.738675302, 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:33.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:33.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:33.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:33.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 121.73857695, 2024-04-18T22:24:33.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:33.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 88.056854997000002, 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.911 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.915 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 88.056200783999998, 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:33.919 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 88.056094088999998, 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:33.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 88.054228179999996, 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.924 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:33.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.928 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 88.053562857000003, 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:33.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 20.511747125999999, 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:33.933 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:33.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:33.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 20.505606628999999, 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.939 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:33.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:33.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:33.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.989300547999999, 2024-04-18T22:24:33.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:33.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:33.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:33.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:33.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.987766623999999, 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:33.950 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:33.955 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 17.981941269, 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:33.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:33.958 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:33.961 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:33.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:33.963 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:33.956+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:34.063 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:34.060+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:34.124 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208238 5.3 5:cd382e9b:::benchmark_data_smithi032_22800_object80124:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.118836+0000", 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0061577180000000004, 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0061752100000000004, 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.125 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208238 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118836+0000", 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118836+0000", 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118837+0000", 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.61e-07 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118840+0000", 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8940000000000002e-06 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118848+0000", 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8609999999999998e-06 2024-04-18T22:24:34.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118855+0000", 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1579999999999999e-06 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120724+0000", 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018694790000000001 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120870+0000", 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000145453 2024-04-18T22:24:34.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121008+0000", 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013801699999999999 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208241 5.7 5:eee22cb4:::benchmark_data_smithi032_22800_object80127:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.119095+0000", 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0058987400000000004, 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0059681509999999997, 2024-04-18T22:24:34.129 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208241 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119095+0000", 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119095+0000", 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119096+0000", 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8100000000000002e-07 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119100+0000", 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2230000000000001e-06 2024-04-18T22:24:34.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119110+0000", 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5840000000000001e-06 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119117+0000", 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.063e-06 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120929+0000", 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018117319999999999 2024-04-18T22:24:34.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121045+0000", 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115826 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121153+0000", 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000108856 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.133 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208242 5.3 5:caea98cb:::benchmark_data_smithi032_22800_object80128:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.120335+0000", 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0046592079999999998, 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047700160000000002, 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208242 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.134 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120335+0000", 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120335+0000", 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120336+0000", 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.33e-07 2024-04-18T22:24:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120341+0000", 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5600000000000001e-06 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120351+0000", 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5580000000000006e-06 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120358+0000", 2024-04-18T22:24:34.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7560000000000002e-06 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122885+0000", 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025270370000000002 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122992+0000", 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107015 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123108+0000", 2024-04-18T22:24:34.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011572499999999999 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208243 5.8 5:14f48c30:::benchmark_data_smithi032_22800_object80129:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.120371+0000", 2024-04-18T22:24:34.138 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0046229859999999999, 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0047749689999999996, 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208243 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120371+0000", 2024-04-18T22:24:34.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120371+0000", 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120372+0000", 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1100000000000001e-07 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120374+0000", 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6010000000000002e-06 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120380+0000", 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8109999999999999e-06 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120385+0000", 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7350000000000004e-06 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120432+0000", 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7080000000000003e-05 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120553+0000", 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012112099999999999 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208244 5.c 5:37ba8bc7:::benchmark_data_smithi032_22800_object80130:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.142 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.120783+0000", 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0042113640000000004, 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044004639999999998, 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208244 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120783+0000", 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120783+0000", 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120783+0000", 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.92e-07 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120787+0000", 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0450000000000001e-06 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120795+0000", 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3610000000000001e-06 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120799+0000", 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6129999999999997e-06 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120842+0000", 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2695999999999999e-05 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120930+0000", 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7843000000000006e-05 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121049+0000", 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011881 2024-04-18T22:24:34.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208245 5.e 5:7432dc5d:::benchmark_data_smithi032_22800_object80131:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.120976+0000", 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0040181119999999999, 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042468039999999999, 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208245 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120976+0000", 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120976+0000", 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120977+0000", 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0130000000000001e-06 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120982+0000", 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2229999999999998e-06 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.120993+0000", 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1161e-05 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121001+0000", 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0120000000000005e-06 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121066+0000", 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4103999999999995e-05 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.121189+0000", 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012322799999999999 2024-04-18T22:24:34.150 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208247 5.c 5:3577bbdc:::benchmark_data_smithi032_22800_object80133:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.122504+0000", 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0024896779999999999, 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027535569999999998, 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208247 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122504+0000", 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122504+0000", 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122505+0000", 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9699999999999995e-07 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122509+0000", 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3359999999999997e-06 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122519+0000", 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2660000000000007e-06 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.122523+0000", 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7219999999999999e-06 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123131+0000", 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00060712099999999996 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123209+0000", 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8175000000000003e-05 2024-04-18T22:24:34.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123278+0000", 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9158000000000004e-05 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208248 5.8 5:191e0051:::benchmark_data_smithi032_22800_object80134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.123311+0000", 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001682809, 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0019875269999999998, 2024-04-18T22:24:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208248 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123311+0000", 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.156 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123311+0000", 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123312+0000", 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.08e-06 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123317+0000", 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9130000000000002e-06 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123328+0000", 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0616e-05 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123336+0000", 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.61e-06 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123398+0000", 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2088000000000006e-05 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.123549+0000", 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000151743 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208250 5.6 5:6d4ed018:::benchmark_data_smithi032_22800_object80136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.124791+0000", 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00020302499999999999, 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00054260599999999995, 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208250 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124791+0000", 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124791+0000", 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124792+0000", 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7599999999999996e-07 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124795+0000", 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5829999999999998e-06 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.161 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124804+0000", 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9449999999999999e-06 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124810+0000", 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0839999999999999e-06 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124863+0000", 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3661e-05 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.162 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124986+0000", 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012290599999999999 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125107+0000", 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012095900000000001 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208251 5.4 5:2ac10b1e:::benchmark_data_smithi032_22800_object80137:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.163 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.124823+0000", 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000170772, 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000548497, 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208251 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.164 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124823+0000", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124823+0000", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124824+0000", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.53e-07 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124826+0000", 2024-04-18T22:24:34.165 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.091e-06 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124831+0000", 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1719999999999998e-06 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124834+0000", 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1769999999999998e-06 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.166 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124869+0000", 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5187e-05 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124953+0000", 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3226000000000006e-05 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:208249 5.0 e62/61 5:0b831f72:::benchmark_data_smithi032_22800_object80135:head v 62'12946, mlcod=62'12943)", 2024-04-18T22:24:34.167 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.125023+0000", 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999709, 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00038647399999999999, 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125023+0000", 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125023+0000", 2024-04-18T22:24:34.168 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125024+0000", 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8999999999999995e-07 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125031+0000", 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5889999999999999e-06 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.169 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125042+0000", 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1207e-05 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125049+0000", 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0559999999999999e-06 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125094+0000", 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5105999999999998e-05 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.170 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.125114+0000", 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0010999999999999e-05 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:208236 5.7 5:edf0471b:::benchmark_data_smithi032_22800_object80122:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:34.118769+0000", 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.006225174, 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0066736790000000001, 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 208236 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118769+0000", 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.172 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118769+0000", 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118770+0000", 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5400000000000001e-07 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118773+0000", 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.27e-06 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.173 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118780+0000", 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7880000000000001e-06 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118789+0000", 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1910000000000006e-06 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118832+0000", 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3275000000000002e-05 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.174 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.118942+0000", 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010979100000000001 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.119042+0000", 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000100505 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:34.124467+0000", 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054248439999999998 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.175 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:34.176 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 12 2024-04-18T22:24:34.176 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:34.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:34.178 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:34.172+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:34.278 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:34.273+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:34.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:34.377+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:34.382 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:34.382 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:34.382 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:34.382 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:34.382 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:34.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:34.481 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:34.477+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:34.570 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:34.571 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:34.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:34.571 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:34.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.571 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:34.571 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:34.572 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.533221602000001, 2024-04-18T22:24:34.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:34.572 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.572 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.573 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.573 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:34.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.573 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:34.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:34.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:34.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:34.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:34.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:34.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:34.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:34.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:34.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:34.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:34.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:34.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:34.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:34.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:34.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:34.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:34.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:34.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:34.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:34.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:34.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.584 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:34.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:34.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.584 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:34.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:34.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:34.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:34.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:34.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:34.587 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.587 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.587 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.587 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:34.588 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:34.588 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.528578603, 2024-04-18T22:24:34.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:34.588 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.589 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:34.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:34.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.596 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.519480865999999, 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:34.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:34.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:34.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.510460941000002, 2024-04-18T22:24:34.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:34.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:34.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.494449573000001, 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.610 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.611 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:34.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:34.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.616 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 31.491646153000001, 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.617 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:34.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:34.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:34.623 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6821471890000002, 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:34.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:34.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.629 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6804809829999998, 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.630 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:34.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:34.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6795084870000001, 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:34.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:34.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.639 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:34.640 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6781485099999998, 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:34.641 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.642 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.643 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.646 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6755813800000001, 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:34.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6745267410000002, 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.653 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:34.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:34.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6736848010000003, 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6718716909999998, 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:34.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:34.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.669 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6716846400000001, 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.671 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:34.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.675 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6714459189999999, 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:34.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.682 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6712567140000001, 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:34.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.687 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6711451770000001, 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.689 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.692 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:34.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.670968352, 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.695 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:34.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 3.6687926270000002, 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:34.700 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:34.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:34.705 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:24:34.707 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:34.708 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:34.704+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:34.808 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:34.805+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:34.866 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:34.866 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:34.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.866 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:210012 5.d 5:b8f5309b:::benchmark_data_smithi032_22800_object81898:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:34.866 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.862367+0000", 2024-04-18T22:24:34.866 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0041777239999999998, 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0042036469999999996, 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 210012 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862367+0000", 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862367+0000", 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862368+0000", 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1399999999999995e-07 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862373+0000", 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9699999999999998e-06 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862388+0000", 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.484e-05 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862397+0000", 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8259999999999999e-06 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862458+0000", 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1075999999999996e-05 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.862539+0000", 2024-04-18T22:24:34.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1126999999999998e-05 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:210016 5.1 5:893d2f8f:::benchmark_data_smithi032_22800_object81902:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.863181+0000", 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003363937, 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034691019999999999, 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:34.871 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 210016 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863181+0000", 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863181+0000", 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863182+0000", 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4300000000000001e-07 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863187+0000", 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6319999999999999e-06 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863197+0000", 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.842e-06 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863203+0000", 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3049999999999998e-06 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863302+0000", 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8905000000000003e-05 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863435+0000", 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000133535 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863515+0000", 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9629999999999995e-05 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.865275+0000", 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001760094 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:210010 5.7 e63/23 5:e7c58599:::benchmark_data_smithi032_22800_object81896:head v 63'13211, mlcod=63'13208)", 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.863328+0000", 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003217012, 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0034018780000000001, 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:34.876 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863328+0000", 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863328+0000", 2024-04-18T22:24:34.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863329+0000", 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7000000000000005e-07 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863335+0000", 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2670000000000003e-06 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863344+0000", 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9290000000000004e-06 2024-04-18T22:24:34.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863351+0000", 2024-04-18T22:24:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.579e-06 2024-04-18T22:24:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863419+0000", 2024-04-18T22:24:34.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8773e-05 2024-04-18T22:24:34.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.863439+0000", 2024-04-18T22:24:34.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9411000000000001e-05 2024-04-18T22:24:34.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.882 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.882 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:210015 5.7 e63/23 5:e13bd3e8:::benchmark_data_smithi032_22800_object81901:head v 63'13212, mlcod=63'13208)", 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.864639+0000", 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0019059179999999999, 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002148807, 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.883 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864639+0000", 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864639+0000", 2024-04-18T22:24:34.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864640+0000", 2024-04-18T22:24:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9699999999999995e-07 2024-04-18T22:24:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864649+0000", 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5390000000000007e-06 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864660+0000", 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1343e-05 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864667+0000", 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9129999999999997e-06 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.865096+0000", 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042931499999999999 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.865109+0000", 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2567e-05 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.888 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:210019 5.0 5:092c88ca:::benchmark_data_smithi032_22800_object81905:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.864709+0000", 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018366739999999999, 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021408579999999998, 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.889 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 210019 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864709+0000", 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864709+0000", 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864709+0000", 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.85e-07 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864715+0000", 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7039999999999999e-06 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864725+0000", 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0363999999999999e-05 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864731+0000", 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.054e-06 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864789+0000", 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7927999999999998e-05 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864892+0000", 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010296799999999999 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.864995+0000", 2024-04-18T22:24:34.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000102866 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:210020 5.7 e63/23 5:e7665702:::benchmark_data_smithi032_22800_object81906:head v 63'13213, mlcod=63'13208)", 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.866143+0000", 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00040220300000000001, 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000779147, 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:34.894 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866143+0000", 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866143+0000", 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866144+0000", 2024-04-18T22:24:34.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7199999999999998e-07 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866151+0000", 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5610000000000001e-06 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866163+0000", 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1725999999999999e-05 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866170+0000", 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3000000000000004e-06 2024-04-18T22:24:34.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866380+0000", 2024-04-18T22:24:34.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021010899999999999 2024-04-18T22:24:34.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866399+0000", 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8226000000000001e-05 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:210022 5.0 5:012abb7d:::benchmark_data_smithi032_22800_object81908:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:34.866699+0000", 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.999846, 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000284515, 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:34.899 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 210022 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866699+0000", 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:34.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866699+0000", 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866700+0000", 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7400000000000003e-07 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866704+0000", 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7510000000000002e-06 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866711+0000", 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4270000000000002e-06 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866716+0000", 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7690000000000004e-06 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866760+0000", 2024-04-18T22:24:34.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4585000000000001e-05 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866846+0000", 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.5690000000000001e-05 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:34.866930+0000", 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4284999999999997e-05 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:34.903 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:34.904 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 7 2024-04-18T22:24:34.904 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:34.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:34.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:34.945+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:35.010 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:35.004+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:35.103 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:35.104 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:35.104 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:35.104 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:35.104 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:35.111 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:35.108+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:35.125 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:35.212 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:35.208+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.164156318, 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.281 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.282 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.283 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:35.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:35.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.286 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.164060472, 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.289 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.290 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.291 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.16400358999999, 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.292 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.294 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.295 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:35.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.297 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.299 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.163943918, 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.300 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:35.301 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:35.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.302 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.302 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.302 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:35.302 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:35.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.303 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:35.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:35.305 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.163885956, 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.306 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.307 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.308 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.309 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.310 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.163666264, 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.312 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:35.313 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.314 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.315 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.316 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.317 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.163378643, 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.318 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.319 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:35.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.321 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:35.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:35.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.16322617500001, 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:35.324 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.325 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:35.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:35.327 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:35.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.329 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.161626596, 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:35.330 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:35.331 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.332 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.333 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:35.334 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.335 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 123.161528244, 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:35.336 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:35.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.339 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:35.340 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 89.479806291000003, 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:35.342 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.343 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:35.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.345 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 89.479152077999998, 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:35.346 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:35.348 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:35.349 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.350 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:35.351 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 89.479045382999999, 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.353 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.354 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.355 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:35.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.357 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.358 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:35.359 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 89.477179473999996, 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.361 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:35.362 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.363 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.364 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:35.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.366 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:35.367 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:35.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.367 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.367 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.367 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:35.368 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.369 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.369 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.369 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.369 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:35.370 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:35.370 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 89.476514151000003, 2024-04-18T22:24:35.370 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:35.370 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.370 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.371 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.372 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:35.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.374 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.375 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.376 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.93469842, 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.377 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:35.378 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.379 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.380 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.381 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:35.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:35.383 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.928557923, 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.384 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.385 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.386 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.387 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:35.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 19.412251842, 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.389 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.390 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.391 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.393 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:35.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 19.410717918, 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.397 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:35.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:35.399 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.400 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.401 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 19.404892563000001, 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:35.402 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.403 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:35.404 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:35.405 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.406 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.407 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.408 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.408 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:35.408 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:35.408 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:35.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:35.410 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:35.404+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:35.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:35.510 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:35.505+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:35.565 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211756 5.2 5:49bced84:::benchmark_data_smithi032_22800_object83642:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.563374+0000", 2024-04-18T22:24:35.565 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001636504, 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001659754, 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211756 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563374+0000", 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563374+0000", 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563375+0000", 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563377+0000", 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6170000000000001e-06 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563385+0000", 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0469999999999996e-06 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563391+0000", 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9859999999999999e-06 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563947+0000", 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055670700000000004 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564078+0000", 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000130777 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564199+0000", 2024-04-18T22:24:35.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120809 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211757 5.c 5:3bfd7dff:::benchmark_data_smithi032_22800_object83643:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.563864+0000", 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001147302, 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012495780000000001, 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211757 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563864+0000", 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563864+0000", 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563865+0000", 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9199999999999999e-07 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563869+0000", 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9589999999999997e-06 2024-04-18T22:24:35.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563878+0000", 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9019999999999994e-06 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563885+0000", 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8870000000000003e-06 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563932+0000", 2024-04-18T22:24:35.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6854000000000002e-05 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564066+0000", 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013415700000000001 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564187+0000", 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012038499999999999 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.574 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211758 5.8 5:1c3b4cc4:::benchmark_data_smithi032_22800_object83644:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.563900+0000", 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0011105690000000001, 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013040809999999999, 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211758 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563900+0000", 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563900+0000", 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563901+0000", 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.46e-07 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563904+0000", 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9340000000000002e-06 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563911+0000", 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9229999999999996e-06 2024-04-18T22:24:35.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563916+0000", 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5300000000000004e-06 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563962+0000", 2024-04-18T22:24:35.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6208999999999998e-05 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564075+0000", 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011312200000000001 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.579 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211759 5.e 5.31c602de (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.565206+0000", 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9998045, 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5433999999999997e-05, 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211759 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565206+0000", 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565206+0000", 2024-04-18T22:24:35.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565207+0000", 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7599999999999997e-07 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565211+0000", 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1230000000000002e-06 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565221+0000", 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6809999999999993e-06 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565228+0000", 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8850000000000001e-06 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.583 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211743 5.7 5:efbd48bc:::benchmark_data_smithi032_22800_object83629:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.558528+0000", 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0064827340000000004, 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0067775730000000003, 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211743 2024-04-18T22:24:35.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558528+0000", 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558528+0000", 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558529+0000", 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2799999999999995e-07 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558533+0000", 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5020000000000003e-06 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558543+0000", 2024-04-18T22:24:35.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9920000000000002e-06 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558550+0000", 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4649999999999999e-06 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558639+0000", 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9167999999999997e-05 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.558767+0000", 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000127692 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.559069+0000", 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000302405 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562703+0000", 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003634169 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211746 5.c 5:3a231afd:::benchmark_data_smithi032_22800_object83632:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.560232+0000", 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0047785900000000001, 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051232609999999996, 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211746 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560232+0000", 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560232+0000", 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560233+0000", 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3e-07 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560237+0000", 2024-04-18T22:24:35.591 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1520000000000002e-06 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560246+0000", 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.0790000000000007e-06 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560251+0000", 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9320000000000004e-06 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.592 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560286+0000", 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4986e-05 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560359+0000", 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.258e-05 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560602+0000", 2024-04-18T22:24:35.593 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000243129 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563600+0000", 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029979220000000001 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211747 5.8 5:10ea0d6d:::benchmark_data_smithi032_22800_object83633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.560832+0000", 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0041787789999999997, 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0045650990000000004, 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211747 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560832+0000", 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.596 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560832+0000", 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560833+0000", 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6099999999999999e-07 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560838+0000", 2024-04-18T22:24:35.597 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8770000000000001e-06 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560848+0000", 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0263e-05 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.560856+0000", 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4930000000000001e-06 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.598 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561920+0000", 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0010646449999999999 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562043+0000", 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122347 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.599 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211749 5.2 5:484893de:::benchmark_data_smithi032_22800_object83635:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.561196+0000", 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003814394, 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042384199999999997, 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211749 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.600 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561196+0000", 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561196+0000", 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561197+0000", 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.1699999999999997e-07 2024-04-18T22:24:35.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561202+0000", 2024-04-18T22:24:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.561e-06 2024-04-18T22:24:35.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561214+0000", 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1839e-05 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.603 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561221+0000", 2024-04-18T22:24:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7039999999999994e-06 2024-04-18T22:24:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.604 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561711+0000", 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00049011400000000004 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561862+0000", 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00015023099999999999 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561978+0000", 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116698 2024-04-18T22:24:35.605 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211751 5.3 5:c2681773:::benchmark_data_smithi032_22800_object83637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.561764+0000", 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032466470000000001, 2024-04-18T22:24:35.606 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037078599999999999, 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211751 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.607 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561764+0000", 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561764+0000", 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561765+0000", 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9299999999999997e-07 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561769+0000", 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7919999999999999e-06 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561776+0000", 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7850000000000003e-06 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561780+0000", 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5299999999999998e-06 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.609 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561829+0000", 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9304000000000001e-05 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561945+0000", 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115648 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562053+0000", 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010768300000000001 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.610 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211752 5.3 5:c7e87650:::benchmark_data_smithi032_22800_object83638:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.561797+0000", 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032139460000000001, 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037139310000000002, 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211752 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561797+0000", 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561797+0000", 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.612 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561798+0000", 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7100000000000001e-07 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561800+0000", 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5069999999999999e-06 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561806+0000", 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3740000000000003e-06 2024-04-18T22:24:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561812+0000", 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3550000000000004e-06 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563956+0000", 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002144188 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.614 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564049+0000", 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2955999999999996e-05 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.564128+0000", 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9005000000000007e-05 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211753 5.a 5:58b40ad0:::benchmark_data_smithi032_22800_object83639:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.561820+0000", 2024-04-18T22:24:35.615 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0031911639999999998, 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037476699999999998, 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211753 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561820+0000", 2024-04-18T22:24:35.616 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561820+0000", 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.617 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561820+0000", 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8100000000000003e-07 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561822+0000", 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2000000000000001e-06 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561829+0000", 2024-04-18T22:24:35.618 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6069999999999996e-06 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561834+0000", 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2580000000000003e-06 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561878+0000", 2024-04-18T22:24:35.619 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3309000000000002e-05 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.561983+0000", 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000105711 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211754 5.3 5:c95fb8a0:::benchmark_data_smithi032_22800_object83640:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.562457+0000", 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025538280000000002, 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0031541009999999999, 2024-04-18T22:24:35.620 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211754 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562457+0000", 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.621 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562457+0000", 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562458+0000", 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8299999999999996e-07 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562462+0000", 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3850000000000004e-06 2024-04-18T22:24:35.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562473+0000", 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0973e-05 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.562480+0000", 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5760000000000002e-06 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565214+0000", 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027341679999999999 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565364+0000", 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000149942 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.624 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.565482+0000", 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011848100000000001 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:211755 5.3 5.aa235893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:35.563342+0000", 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0016684849999999999, 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023207089999999998, 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:35.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 211755 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563342+0000", 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.626 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563342+0000", 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563343+0000", 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8500000000000001e-07 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563347+0000", 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4699999999999998e-06 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.627 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563354+0000", 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7670000000000003e-06 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:35.563361+0000", 2024-04-18T22:24:35.628 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6390000000000003e-06 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 13 2024-04-18T22:24:35.629 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:35.630 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:35.631 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:35.625+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:35.631 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:35.631 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:35.632 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:35.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:35.633 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:35.633 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:35.633 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:35.731 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:35.728+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:35.786 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:35.786 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:35.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:35.847 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:35.844+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:35.933 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:35.929+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:36.003 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:36.004 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:36.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:36.067 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:36.065+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:36.134 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:36.132+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:36.185 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:36.185 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:36.208 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:36.251 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:36.249+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:36.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:36.333+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:36.411 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:36.412 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:36.433 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:36.475 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:36.473+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:36.537 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:36.536+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:36.630 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:36.630 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:36.639 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:36.640+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:36.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:36.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:24:36.739 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:36.740+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:36.801 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:36.801 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:36.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:36.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:36.837+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:36.942 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:36.940+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:37.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:37.080 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:37.080+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:37.143 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.141+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.231 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:37.232 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:37.244 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.241+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.252 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:37.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.341+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.401 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:37.401 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:37.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:37.467 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.465+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.547 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.545+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.617 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:37.617 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:37.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:37.637 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:37.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.677+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.750 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.745+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:37.801 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:37.802 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:37.802 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:37.819 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:37.820 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:37.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:37.822 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.71994017599999, 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.71984433, 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.71978744800001, 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.719727776, 2024-04-18T22:24:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.719669814, 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.719450122, 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.719162501, 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.719010033, 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.717410454, 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 125.71731210199999, 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.035590149000001, 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:37.888 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.889 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:37.890 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:37.891 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.892 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.034935935999997, 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.893 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.894 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.896 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.034829240999997, 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:37.897 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:37.898 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:37.899 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:37.900 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.032963331999994, 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:37.901 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.902 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.903 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.904 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 92.032298009000002, 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:37.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 24.490482278000002, 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.909 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.914 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 24.484341781000001, 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:37.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:37.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:37.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:37.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:37.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.968035700000001, 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.966501776000001, 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.926 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:37.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:37.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.932 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 21.960676420999999, 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.933 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:37.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:37.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:37.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:37.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:37.938 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:37.939 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:24:37.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:37.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:37.937+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.041 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.037+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.095 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:38.095 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:38.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.095 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218300 5.7 5:eb19b2b5:::benchmark_data_smithi032_22800_object90186:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.095 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.089340+0000", 2024-04-18T22:24:38.095 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0051777259999999997, 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052061499999999997, 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218300 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089340+0000", 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089340+0000", 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089340+0000", 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.27e-07 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089342+0000", 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3889999999999999e-06 2024-04-18T22:24:38.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089347+0000", 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6890000000000003e-06 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089350+0000", 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.269e-06 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089400+0000", 2024-04-18T22:24:38.098 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9963999999999998e-05 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089522+0000", 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000121314 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089666+0000", 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014405599999999999 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:218295 5.1 e64/49 5:8644623f:::benchmark_data_smithi032_22800_object90181:head v 64'13542, mlcod=64'13540)", 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.089516+0000", 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.005001368, 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0051150500000000003, 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089516+0000", 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089516+0000", 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089517+0000", 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8699999999999995e-07 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089520+0000", 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8929999999999998e-06 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089528+0000", 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.96e-06 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089533+0000", 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2760000000000002e-06 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089599+0000", 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6036000000000003e-05 2024-04-18T22:24:38.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089621+0000", 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2265000000000001e-05 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:218299 5.1 e64/49 5:8ce72708:::benchmark_data_smithi032_22800_object90185:head v 64'13543, mlcod=64'13540)", 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.090946+0000", 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0035711240000000002, 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037466890000000001, 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.090946+0000", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.090946+0000", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.090947+0000", 2024-04-18T22:24:38.104 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5199999999999996e-07 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.090955+0000", 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2830000000000001e-06 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.090963+0000", 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.977e-06 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.090968+0000", 2024-04-18T22:24:38.105 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2580000000000003e-06 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091017+0000", 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8822000000000001e-05 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091037+0000", 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9925999999999999e-05 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218301 5.7 5:e99876b3:::benchmark_data_smithi032_22800_object90187:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.091922+0000", 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025952890000000002, 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028347170000000001, 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218301 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091922+0000", 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091922+0000", 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091923+0000", 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0550000000000001e-06 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091928+0000", 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2560000000000002e-06 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091941+0000", 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2238e-05 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091949+0000", 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9699999999999999e-06 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.109 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092000+0000", 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0884000000000003e-05 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092125+0000", 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012564100000000001 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092237+0000", 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011163300000000001 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218302 5.b 5:dd5ed996:::benchmark_data_smithi032_22800_object90188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.091968+0000", 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00254974, 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028677109999999998, 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218302 2024-04-18T22:24:38.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091968+0000", 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091968+0000", 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091968+0000", 2024-04-18T22:24:38.112 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6199999999999998e-07 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091971+0000", 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8389999999999998e-06 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091979+0000", 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9729999999999997e-06 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091985+0000", 2024-04-18T22:24:38.113 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8950000000000003e-06 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092026+0000", 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1025000000000001e-05 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092158+0000", 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000131896 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218303 5.a 5:58b2c704:::benchmark_data_smithi032_22800_object90189:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.091999+0000", 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025182500000000001, 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002908927, 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218303 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091999+0000", 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091999+0000", 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092000+0000", 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2e-07 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092002+0000", 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.689e-06 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092009+0000", 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0790000000000003e-06 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092015+0000", 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4489999999999998e-06 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092075+0000", 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0439000000000002e-05 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092204+0000", 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000128264 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.118 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218304 5.7 5:e0999628:::benchmark_data_smithi032_22800_object90190:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.092093+0000", 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0024240429999999999, 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028784129999999998, 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218304 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.119 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092093+0000", 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092093+0000", 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092094+0000", 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3600000000000002e-07 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.120 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092101+0000", 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7689999999999999e-06 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092111+0000", 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7419999999999992e-06 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092120+0000", 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9940000000000006e-06 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.121 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093963+0000", 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018430020000000001 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094081+0000", 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011783200000000001 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094187+0000", 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010631 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:218305 5.1 e64/49 5:89828b79:::benchmark_data_smithi032_22800_object90191:head v 64'13544, mlcod=64'13540)", 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.093391+0000", 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0011260910000000001, 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001659235, 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093391+0000", 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093391+0000", 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093392+0000", 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9900000000000001e-07 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093396+0000", 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2459999999999997e-06 2024-04-18T22:24:38.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093403+0000", 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1339999999999998e-06 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093409+0000", 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0569999999999998e-06 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093461+0000", 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1413999999999999e-05 2024-04-18T22:24:38.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093480+0000", 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9505e-05 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:218300 5.7 e64/23)", 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.093612+0000", 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00090531300000000004, 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0014890579999999999, 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093612+0000", 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093612+0000", 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.127 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093612+0000", 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.22e-07 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093614+0000", 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9910000000000001e-06 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093620+0000", 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5969999999999999e-06 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.128 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.093625+0000", 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2059999999999998e-06 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:218301 5.7 e64/23)", 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.094472+0000", 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4.5025999999999999e-05, 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000670508, 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.129 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094472+0000", 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094472+0000", 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094473+0000", 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0299999999999999e-07 2024-04-18T22:24:38.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094478+0000", 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6650000000000002e-06 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.131 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094498+0000", 2024-04-18T22:24:38.133 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0628999999999998e-05 2024-04-18T22:24:38.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.133 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.133 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094505+0000", 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7089999999999996e-06 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218306 5.3 5:c236a0a9:::benchmark_data_smithi032_22800_object90192:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.094604+0000", 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999137, 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00058737800000000005, 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.134 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218306 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094604+0000", 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094604+0000", 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.135 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094605+0000", 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1200000000000001e-06 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094610+0000", 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1050000000000003e-06 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094628+0000", 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7676000000000002e-05 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.136 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094635+0000", 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1550000000000001e-06 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094682+0000", 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7154000000000003e-05 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094780+0000", 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8350999999999994e-05 2024-04-18T22:24:38.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.094842+0000", 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1731999999999998e-05 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218310 5.4 5:201facb7:::benchmark_data_smithi032_22800_object90196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.095054+0000", 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9994636, 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205035, 2024-04-18T22:24:38.138 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "reached pg", 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218310 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095054+0000", 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.139 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095054+0000", 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095055+0000", 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2699999999999999e-07 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095059+0000", 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1359999999999999e-06 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.140 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095070+0000", 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1532000000000001e-05 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095078+0000", 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5499999999999997e-06 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095134+0000", 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5844999999999998e-05 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.141 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.095263+0000", 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012955700000000001 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218293 5.e 5:79f33bd5:::benchmark_data_smithi032_22800_object90179:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.088986+0000", 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0055313309999999996, 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063349679999999998, 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218293 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.088986+0000", 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.088986+0000", 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.088987+0000", 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1699999999999997e-07 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.088991+0000", 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.318e-06 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.144 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089001+0000", 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4369999999999996e-06 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089008+0000", 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3930000000000002e-06 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089280+0000", 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000272269 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.145 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089385+0000", 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104957 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218297 5.e 5:7a46f933:::benchmark_data_smithi032_22800_object90183:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.089289+0000", 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0052283360000000001, 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060946209999999997, 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:38.146 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218297 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089289+0000", 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089289+0000", 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089289+0000", 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.15e-07 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089291+0000", 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.852e-06 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089296+0000", 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2100000000000003e-06 2024-04-18T22:24:38.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089299+0000", 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0680000000000001e-06 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092360+0000", 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003061299 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.092489+0000", 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012864800000000001 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.149 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:218298 5.6 5:6562aaa0:::benchmark_data_smithi032_22800_object90184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:38.089304+0000", 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0052133029999999999, 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0061437690000000003, 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 218298 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089304+0000", 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089304+0000", 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.151 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089304+0000", 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8099999999999998e-07 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089307+0000", 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4600000000000002e-06 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089313+0000", 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3670000000000002e-06 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.152 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.089319+0000", 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6799999999999998e-06 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091710+0000", 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023913300000000001 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091851+0000", 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014064200000000001 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.153 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:38.091976+0000", 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124981 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 15 2024-04-18T22:24:38.154 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:38.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:38.156 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.153+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.256 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.253+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.303 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:38.303 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:38.304 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:38.304 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:38.304 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:38.320 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:38.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.353+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.458 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.453+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 35.437415428000001, 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:38.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 35.432772429000003, 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.481 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:38.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:38.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:38.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 35.423674691999999, 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:38.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:38.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.493 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 35.414654767000002, 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.494 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:38.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:38.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:38.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 35.398643399000001, 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:38.501 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:38.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:38.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 35.395839979000002, 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.508 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:38.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:38.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:38.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:38.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:38.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5863410150000004, 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.514 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.519 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.584674809, 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:38.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:38.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5837023129999999, 2024-04-18T22:24:38.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:38.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.582342336, 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.531 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.534 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5797752059999999, 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:38.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:38.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:38.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:38.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.541 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.541 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5787205669999995, 2024-04-18T22:24:38.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:38.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:38.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5778786270000005, 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.548 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:38.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:38.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:38.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.576065517, 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.558 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5758784659999998, 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.559 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:38.560 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:38.561 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:38.562 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.563 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5756397450000001, 2024-04-18T22:24:38.564 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:38.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:38.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5754505400000003, 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:38.569 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.570 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5753390029999998, 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.575 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.576 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.577 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5751621780000002, 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.579 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:38.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:38.582 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:38.583 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 7.5729864530000004, 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:38.584 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.585 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.587 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.588 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:38.589 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:38.589 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:38.590 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:38.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.585+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.691 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:38.691+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:38.747 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:38.747 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:38.747 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.747 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:219923 5.5 5:a2924a48:::benchmark_data_smithi032_22800_object91809:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.747 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:38.745063+0000", 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0012664169999999999, 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0012915170000000001, 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:38.748 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 219923 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745063+0000", 2024-04-18T22:24:38.749 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.750 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.750 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:38.750 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745063+0000", 2024-04-18T22:24:38.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.751 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.751 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:38.751 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745064+0000", 2024-04-18T22:24:38.751 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1200000000000002e-07 2024-04-18T22:24:38.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.752 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.752 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:38.752 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745067+0000", 2024-04-18T22:24:38.752 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8870000000000002e-06 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745079+0000", 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1449e-05 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.753 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745087+0000", 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4090000000000003e-06 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745126+0000", 2024-04-18T22:24:38.754 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8970000000000001e-05 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745269+0000", 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014302100000000001 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.755 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:38.756 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745388+0000", 2024-04-18T22:24:38.756 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011864199999999999 2024-04-18T22:24:38.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.756 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:38.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.757 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.757 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:219924 5.1 5:8b21ee10:::benchmark_data_smithi032_22800_object91810:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.757 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:38.745103+0000", 2024-04-18T22:24:38.758 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0012257349999999999, 2024-04-18T22:24:38.758 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001328086, 2024-04-18T22:24:38.758 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:38.758 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 219924 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745103+0000", 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745103+0000", 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745104+0000", 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0900000000000002e-07 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745106+0000", 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5670000000000002e-06 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745114+0000", 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0160000000000003e-06 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745119+0000", 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5230000000000003e-06 2024-04-18T22:24:38.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745164+0000", 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4561000000000002e-05 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745276+0000", 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011225799999999999 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.762 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745388+0000", 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011207199999999999 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:219926 5.6 e64/19 5:67ae4364:::benchmark_data_smithi032_22800_object91812:head v 64'13638, mlcod=64'13636)", 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:38.745668+0000", 2024-04-18T22:24:38.763 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000660696, 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00083127600000000004, 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745668+0000", 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.764 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745668+0000", 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745669+0000", 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1500000000000004e-07 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745673+0000", 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3919999999999996e-06 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.765 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745689+0000", 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5282e-05 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745694+0000", 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.797e-06 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.766 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745751+0000", 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.7747999999999997e-05 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.745763+0000", 2024-04-18T22:24:38.767 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2146e-05 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:219925 5.3 e64/19 5:c7d5cde0:::benchmark_data_smithi032_22800_object91811:head v 64'14045, mlcod=64'14043)", 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:38.746162+0000", 2024-04-18T22:24:38.768 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000166906, 2024-04-18T22:24:38.769 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00039253500000000001, 2024-04-18T22:24:38.769 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:38.769 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:38.769 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:38.769 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:38.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.770 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:38.770 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746162+0000", 2024-04-18T22:24:38.770 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.770 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.771 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:38.772 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746162+0000", 2024-04-18T22:24:38.772 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:38.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746163+0000", 2024-04-18T22:24:38.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.58e-07 2024-04-18T22:24:38.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:38.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746167+0000", 2024-04-18T22:24:38.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7659999999999998e-06 2024-04-18T22:24:38.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746175+0000", 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0190000000000006e-06 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746182+0000", 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0220000000000001e-06 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746241+0000", 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9296999999999998e-05 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.746263+0000", 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1758000000000001e-05 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:219918 5.9 5:907635e6:::benchmark_data_smithi032_22800_object91804:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:38.743505+0000", 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0028238500000000001, 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003108165, 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 219918 2024-04-18T22:24:38.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743505+0000", 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743505+0000", 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:38.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743506+0000", 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4000000000000001e-07 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743510+0000", 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9559999999999999e-06 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743520+0000", 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5340000000000005e-06 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743525+0000", 2024-04-18T22:24:38.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1240000000000004e-06 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743573+0000", 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8591000000000003e-05 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:38.743656+0000", 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3022999999999994e-05 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.782 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:38.783 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:38.783 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 5 2024-04-18T22:24:38.783 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:38.784 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:38.790 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~64s 2024-04-18T22:24:38.831 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.829+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.889+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:38.936 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:38.956 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:38.999 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:38.997+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:39.094 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:39.089+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:39.154 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:39.154 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.036948322, 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.155 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:39.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:39.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.03685247600001, 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.160 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:39.163 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.164 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.165 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.036795594, 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:39.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:39.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.03673592200001, 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.172 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:39.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:39.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:39.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.03667796000001, 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.178 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:39.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:39.183 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.036458268, 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:39.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:39.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.03617064700001, 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:39.189 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:39.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:39.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.036018179, 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:39.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:39.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:39.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.0344186, 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.200 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:39.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:39.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:39.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:39.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 127.034320248, 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:39.205 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 93.352598295000007, 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.209 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:39.213 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 93.351944082000003, 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:39.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 93.351837387000003, 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:39.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:39.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 93.349971478000001, 2024-04-18T22:24:39.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:39.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.226 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 93.349306154999994, 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.230 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 25.807490424000001, 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 25.801349927, 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.242 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.285043846000001, 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.243 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:39.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:39.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:39.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:39.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.283509922, 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:39.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.253 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 23.277684567000001, 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:39.254 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:39.255 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:39.256 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:39.257 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.258 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:39.259 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:39.260 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:39.261 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.259+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:39.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:39.357+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:39.419 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221622 5.b 5:dcfe7718:::benchmark_data_smithi032_22800_object93508:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.415963+0000", 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0034884170000000002, 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035078470000000001, 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221622 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.415963+0000", 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.415963+0000", 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.415964+0000", 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8999999999999996e-07 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.415967+0000", 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9849999999999998e-06 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.415974+0000", 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0219999999999999e-06 2024-04-18T22:24:39.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.415978+0000", 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5159999999999996e-06 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416032+0000", 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3557000000000002e-05 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416169+0000", 2024-04-18T22:24:39.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013729500000000001 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221626 5.7 5:e4822186:::benchmark_data_smithi032_22800_object93512:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.416140+0000", 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0033118039999999998, 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033738560000000002, 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.424 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221626 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416140+0000", 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416140+0000", 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416141+0000", 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8599999999999998e-07 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416144+0000", 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0180000000000002e-06 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416151+0000", 2024-04-18T22:24:39.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9569999999999996e-06 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416155+0000", 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2649999999999998e-06 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416212+0000", 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6753999999999999e-05 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416334+0000", 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000122457 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416439+0000", 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104955 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419241+0000", 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00280248 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221627 5.a 5:5a89cb76:::benchmark_data_smithi032_22800_object93513:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.416178+0000", 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00327363, 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033777009999999999, 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221627 2024-04-18T22:24:39.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416178+0000", 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416178+0000", 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416179+0000", 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8699999999999995e-07 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416182+0000", 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8590000000000001e-06 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416193+0000", 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1483e-05 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416199+0000", 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7749999999999998e-06 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416254+0000", 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4975999999999997e-05 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416368+0000", 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114233 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.432 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221628 5.6 5:6f042a57:::benchmark_data_smithi032_22800_object93514:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.416213+0000", 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032389210000000001, 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003377452, 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.433 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221628 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416213+0000", 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416213+0000", 2024-04-18T22:24:39.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416213+0000", 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0399999999999996e-07 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416216+0000", 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4609999999999998e-06 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416222+0000", 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2570000000000004e-06 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416227+0000", 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1680000000000003e-06 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416310+0000", 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2577000000000006e-05 2024-04-18T22:24:39.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416420+0000", 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109601 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416526+0000", 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106351 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221629 5.e 5:71e13989:::benchmark_data_smithi032_22800_object93515:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.416243+0000", 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0032086910000000001, 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0033862990000000002, 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221629 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.438 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416243+0000", 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416243+0000", 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416244+0000", 2024-04-18T22:24:39.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0800000000000005e-07 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416246+0000", 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4990000000000001e-06 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416252+0000", 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.348e-06 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.416258+0000", 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1039999999999998e-06 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418320+0000", 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002062356 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418437+0000", 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117358 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:221623 5.1 e64/49 5:8fecc091:::benchmark_data_smithi032_22800_object93509:head v 64'13756, mlcod=64'13754)", 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.418355+0000", 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00109658, 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013083109999999999, 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418355+0000", 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418355+0000", 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418356+0000", 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3900000000000004e-07 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418361+0000", 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2170000000000002e-06 2024-04-18T22:24:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418369+0000", 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8529999999999992e-06 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418376+0000", 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5409999999999997e-06 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418425+0000", 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9076000000000002e-05 2024-04-18T22:24:39.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418445+0000", 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0131000000000001e-05 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:221624 5.5 e64/49 5:aa8ef76e:::benchmark_data_smithi032_22800_object93510:head v 64'13696, mlcod=64'13693)", 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.418394+0000", 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001057299, 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0013129490000000001, 2024-04-18T22:24:39.445 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418394+0000", 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418394+0000", 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418395+0000", 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8500000000000002e-07 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418399+0000", 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6830000000000001e-06 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418406+0000", 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2810000000000003e-06 2024-04-18T22:24:39.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418412+0000", 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2960000000000004e-06 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418447+0000", 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4409000000000002e-05 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418462+0000", 2024-04-18T22:24:39.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5931e-05 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221633 5.a 5:59f68c57:::benchmark_data_smithi032_22800_object93519:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.418908+0000", 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00054330800000000005, 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00084565900000000002, 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.449 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221633 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418908+0000", 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418908+0000", 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418910+0000", 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0640000000000001e-06 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418915+0000", 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5160000000000002e-06 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418928+0000", 2024-04-18T22:24:39.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2548000000000001e-05 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418934+0000", 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1199999999999999e-06 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.418988+0000", 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.4524000000000002e-05 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419098+0000", 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000110226 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221634 5.a 5.b70c7f0a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.419072+0000", 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.000380033, 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000715459, 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221634 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419072+0000", 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419072+0000", 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419072+0000", 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3000000000000001e-07 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419075+0000", 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0290000000000001e-06 2024-04-18T22:24:39.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419086+0000", 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1079e-05 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419093+0000", 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9929999999999998e-06 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:221635 5.7 5.4cb49257 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.419692+0000", 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997602, 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013498699999999999, 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 221635 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.457 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419692+0000", 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419692+0000", 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419692+0000", 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7899999999999997e-07 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419697+0000", 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3100000000000002e-06 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419707+0000", 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0626e-05 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419716+0000", 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3930000000000008e-06 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:221632 5.5 e64/49)", 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:39.419812+0000", 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.99964, 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7124999999999999e-05, 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:39.460 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419812+0000", 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419812+0000", 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419812+0000", 2024-04-18T22:24:39.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.44e-07 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419817+0000", 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.164e-06 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419828+0000", 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0684e-05 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:39.419835+0000", 2024-04-18T22:24:39.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4250000000000003e-06 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 11 2024-04-18T22:24:39.463 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:39.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:39.465 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.463+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.566 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.563+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.659 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:39.659 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:39.659 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:39.659 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:39.660 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:39.667 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.667+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:39.770 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.767+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:39.870 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.871+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:39.874 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.836609498999998, 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:39.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:39.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:39.881 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.8319665, 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:39.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:39.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.888 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.822868763000002, 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.889 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:39.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:39.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:39.892 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:39.893 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.813848837999998, 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.895 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.897 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.898 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.900 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.797837469999997, 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:39.902 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:39.903 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:39.904 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:39.905 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 36.795034049999998, 2024-04-18T22:24:39.907 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.910 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.912 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:39.913 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9855350860000005, 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:39.914 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:39.915 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:39.916 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:39.917 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9838688799999993, 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.919 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:39.920 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:39.921 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:39.923 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.982896384, 2024-04-18T22:24:39.924 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:39.925 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:39.929 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9815364070000001, 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:39.930 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.931 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.932 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.933 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.934 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9789692770000009, 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:39.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.938 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:39.940 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9779146379999997, 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.941 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.942 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.943 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:39.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:39.945 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9770726980000006, 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.947 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:39.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.949 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.950 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9752595880000001, 2024-04-18T22:24:39.952 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.953 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.954 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:39.956 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9750725369999991, 2024-04-18T22:24:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:39.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.961 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9748338160000003, 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.964 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:39.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9746446110000004, 2024-04-18T22:24:39.968 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.971 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.972 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:39.972 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:39.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.972 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.973 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:39.973 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:39.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.974 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.974533074, 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.975 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:39.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.978 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.978 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:39.978 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:39.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9743562489999995, 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.982 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.983 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.984 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.985 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 8.9721805240000005, 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:39.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:39.987 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:39.988 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:39.989 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:39.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:39.991 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:39.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:24:39.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:39.994 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:39.991+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:39.995 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:39.995 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:39.996 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:40.014 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:40.017 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:40.018 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:40.094 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:40.089+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:40.152 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:40.152 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:40.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:40.215 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:40.213+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:40.295 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:40.295+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:40.372 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:40.372 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:40.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:40.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:40.393+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:40.497 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:40.495+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:40.597 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:40.597 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:40.598 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:40.595+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:40.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:40.699 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:40.699+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:40.770 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:40.770 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:40.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:40.831 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:40.831+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:40.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:40.897+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:40.958 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:24:40.980 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:40.980 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:41.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:41.002 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:40.997+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:41.102 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:41.103+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:41.155 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:41.156 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:41.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:41.223 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:41.223+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:41.305 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:41.301+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:41.330 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:41.330 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:41.351 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:41.439 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:41.437+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:41.506 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:41.501+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:41.545 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:41.546 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:41.567 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:41.607 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:41.607+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:41.708 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:41.705+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:41.773 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:41.773 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:41.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:41.839 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:41.839+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:41.912 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:41.911+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:41.942 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:41.943 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:41.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:42.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:42.047 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:42.047+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:42.116 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:42.113+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:42.147 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:42.147 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:42.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:42.182 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:42.201 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:42.204 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:42.205 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:42.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:42.213+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:42.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:42.317+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:42.359 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.323085714000001, 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.360 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:42.361 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.362 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.364 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.366 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.318442715000003, 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.367 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.369 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.370 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:42.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:42.372 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:42.373 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.309344977999999, 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.374 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.375 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:42.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.377 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:42.378 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.379 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.300325053000002, 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.381 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.382 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:42.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:42.384 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.385 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.386 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.284313685000001, 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.387 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.388 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.389 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:42.390 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.391 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.392 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:42.393 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 39.281510265000001, 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.394 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.395 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.396 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.397 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:42.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.399 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.472011301, 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.401 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.402 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.403 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.404 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.470345095000001, 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:42.407 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:42.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:42.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.469372599, 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:42.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.414 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.415 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.468012622, 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.416 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.418 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:42.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.465445492000001, 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.422 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:42.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:42.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:42.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:42.429 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.464390852999999, 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:42.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:42.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.435 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.463548913, 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.461735803, 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.461548752000001, 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:42.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.461310031, 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.456 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.461120826, 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:42.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:42.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.461009289, 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.462 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:42.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.460832463999999, 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.467 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:42.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 11.458656739, 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:42.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:42.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:42.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:42.477 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:42.479 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:42.479 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:42.475+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:42.579 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:42.579+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:42.634 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:229765 5.1 5:81e8384c:::benchmark_data_smithi032_22800_object101651:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.632729+0000", 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0025373470000000001, 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002561256, 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:42.635 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 229765 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632729+0000", 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632729+0000", 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.636 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632730+0000", 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5099999999999998e-07 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632735+0000", 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.13e-06 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632745+0000", 2024-04-18T22:24:42.637 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0577000000000001e-05 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632752+0000", 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5400000000000001e-06 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632813+0000", 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1339000000000003e-05 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.638 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.632949+0000", 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000135759 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633082+0000", 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000132748 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:229766 5.5 5:a7fe1a35:::benchmark_data_smithi032_22800_object101652:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.633001+0000", 2024-04-18T22:24:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0022655650000000002, 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023757639999999998, 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 229766 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633001+0000", 2024-04-18T22:24:42.640 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633001+0000", 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633001+0000", 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0500000000000003e-07 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.641 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633005+0000", 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0389999999999998e-06 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633016+0000", 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1061e-05 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633021+0000", 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7369999999999997e-06 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.642 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633075+0000", 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3786999999999998e-05 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633214+0000", 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013886200000000001 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633349+0000", 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000134967 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.643 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:229760 5.3 e65/19 5:c1892af8:::benchmark_data_smithi032_22800_object101646:head v 65'14658, mlcod=65'14656)", 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.633457+0000", 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018092749999999999, 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020005890000000001, 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.644 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633457+0000", 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633457+0000", 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633457+0000", 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6499999999999999e-07 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.645 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633488+0000", 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.1028999999999997e-05 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633500+0000", 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1562999999999999e-05 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633505+0000", 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4240000000000001e-06 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.646 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633569+0000", 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3793000000000004e-05 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633589+0000", 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9414000000000001e-05 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:229761 5.6 e65/19 5:670a48d8:::benchmark_data_smithi032_22800_object101647:head v 65'14253, mlcod=65'14250)", 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.633533+0000", 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0017329610000000001, 2024-04-18T22:24:42.647 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019796290000000001, 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633533+0000", 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633533+0000", 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.648 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633534+0000", 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8599999999999999e-07 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633538+0000", 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1919999999999998e-06 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633544+0000", 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1619999999999996e-06 2024-04-18T22:24:42.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633549+0000", 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7369999999999997e-06 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633591+0000", 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1978999999999999e-05 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633610+0000", 2024-04-18T22:24:42.650 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9281999999999999e-05 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:229763 5.c e65/19 5:3015f8cf:::benchmark_data_smithi032_22800_object101649:head v 65'14316, mlcod=65'14310)", 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.633562+0000", 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001704381, 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019827270000000001, 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.651 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633562+0000", 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633562+0000", 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633562+0000", 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6699999999999999e-07 2024-04-18T22:24:42.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633566+0000", 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0740000000000003e-06 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633572+0000", 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0499999999999997e-06 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633578+0000", 2024-04-18T22:24:42.653 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.446e-06 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633638+0000", 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0590000000000001e-05 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633664+0000", 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5901000000000002e-05 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:229768 5.f 5:fd18342c:::benchmark_data_smithi032_22800_object101654:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.633854+0000", 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0014124470000000001, 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001733334, 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 229768 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.655 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633854+0000", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633854+0000", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633855+0000", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9100000000000002e-07 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633859+0000", 2024-04-18T22:24:42.656 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.2640000000000002e-06 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633871+0000", 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1793e-05 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633877+0000", 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.686e-06 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.633933+0000", 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6045000000000003e-05 2024-04-18T22:24:42.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634061+0000", 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012768600000000001 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:229762 5.6 e65/19 5:6542fed6:::benchmark_data_smithi032_22800_object101648:head v 65'14254, mlcod=65'14250)", 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.634204+0000", 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0010618419999999999, 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001449444, 2024-04-18T22:24:42.658 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634204+0000", 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634204+0000", 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.659 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634205+0000", 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9299999999999998e-07 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634209+0000", 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4549999999999997e-06 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634218+0000", 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3040000000000005e-06 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.660 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.634224+0000", 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2979999999999997e-06 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635399+0000", 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011756729999999999 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635418+0000", 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8783e-05 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.661 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:229769 5.6 e65/19)", 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:42.635585+0000", 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 4294967295.999681, 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000108979, 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.662 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635585+0000", 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635585+0000", 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635586+0000", 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8000000000000005e-07 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.663 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635598+0000", 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2662e-05 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635616+0000", 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7201999999999999e-05 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:42.635624+0000", 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3979999999999999e-06 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.664 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:42.665 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.665 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:42.665 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:42.665 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 8 2024-04-18T22:24:42.665 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:42.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:42.711 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:42.709+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:42.781 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:42.779+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:42.820 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:42.820 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:42.820 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:42.821 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:42.821 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:42.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:42.882 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:42.879+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:42.983 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:42.983+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:42.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92977807299999, 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.929682227, 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92962534500001, 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92956567300001, 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:43.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:43.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92950771100001, 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:43.068 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:43.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:43.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.072 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.073 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92928801900001, 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.929000398, 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.079 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:43.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:43.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:43.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92884792999999, 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:43.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.089 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.927248351, 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.090 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.091 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.092 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:43.094 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.095 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 130.92714999899999, 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.096 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:43.097 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:43.098 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.099 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 97.245428046000001, 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.100 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:43.101 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.102 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.103 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 97.244773832999996, 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.104 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.105 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:43.106 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:43.107 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 97.244667137999997, 2024-04-18T22:24:43.108 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.109 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:43.110 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:43.111 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.112 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 97.242801228999994, 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.113 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:43.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:43.115 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:43.116 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 97.242135906000001, 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.117 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:43.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:43.119 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.120 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.700320175000002, 2024-04-18T22:24:43.121 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:43.122 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:43.123 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.124 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.125 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 29.694179678000001, 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.127 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.128 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:43.129 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.130 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.131 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.132 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.177873597000001, 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:43.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.134 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.135 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.136 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.137 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:43.138 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.176339673000001, 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:43.140 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:43.141 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.142 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.143 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 27.170514317999999, 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:43.144 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:43.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:43.149 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:43.151 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:43.149+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:43.151 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:43.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:43.249+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231379 5.c 5:3d0484e1:::benchmark_data_smithi032_22800_object103265:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.295179+0000", 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0043457870000000003, 2024-04-18T22:24:43.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043693489999999998, 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231379 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295179+0000", 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295179+0000", 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295179+0000", 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7599999999999996e-07 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295184+0000", 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2429999999999999e-06 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295190+0000", 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7240000000000003e-06 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295195+0000", 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6070000000000001e-06 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295244+0000", 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9320999999999998e-05 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295407+0000", 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000162623 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295525+0000", 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000117821 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.304 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231380 5.6 5:6807a589:::benchmark_data_smithi032_22800_object103266:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.295391+0000", 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0041331019999999996, 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042327980000000003, 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231380 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295391+0000", 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295391+0000", 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295392+0000", 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3499999999999996e-07 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295395+0000", 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.641e-06 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295405+0000", 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1989999999999995e-06 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295409+0000", 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3579999999999996e-06 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295460+0000", 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0590000000000002e-05 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295596+0000", 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013663299999999999 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295725+0000", 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012842499999999999 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:231372 5.5 e65/49 5:a82d1fc3:::benchmark_data_smithi032_22800_object103258:head v 65'14297, mlcod=65'14293)", 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.295557+0000", 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0039674920000000004, 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0041348349999999999, 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295557+0000", 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295557+0000", 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295557+0000", 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9900000000000001e-07 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295561+0000", 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5109999999999999e-06 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295568+0000", 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000004e-06 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295572+0000", 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6959999999999998e-06 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295612+0000", 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9521000000000003e-05 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295630+0000", 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8756000000000001e-05 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.312 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:231381 5.1 e65/49 5:84c6bc6f:::benchmark_data_smithi032_22800_object103267:head v 65'14401, mlcod=65'14398)", 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.295759+0000", 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0037656999999999999, 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039860160000000002, 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295759+0000", 2024-04-18T22:24:43.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295759+0000", 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295759+0000", 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.3900000000000005e-07 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295763+0000", 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.422e-06 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295773+0000", 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0635999999999999e-05 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.295779+0000", 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6749999999999999e-06 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297541+0000", 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017621920000000001 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297569+0000", 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7419999999999998e-05 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231382 5.b 5:dfe3bfb9:::benchmark_data_smithi032_22800_object103268:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.316 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.296971+0000", 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0025529630000000001, 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028313230000000002, 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231382 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296971+0000", 2024-04-18T22:24:43.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296971+0000", 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296972+0000", 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2099999999999996e-07 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296976+0000", 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7529999999999999e-06 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296983+0000", 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2779999999999996e-06 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296988+0000", 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0490000000000003e-06 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297039+0000", 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0467999999999999e-05 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297125+0000", 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6167000000000004e-05 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231384 5.b 5:db4cf91e:::benchmark_data_smithi032_22800_object103270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.297151+0000", 2024-04-18T22:24:43.320 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023738000000000001, 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027127739999999998, 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231384 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297151+0000", 2024-04-18T22:24:43.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297151+0000", 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297151+0000", 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0200000000000001e-07 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297155+0000", 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9040000000000002e-06 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297163+0000", 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2379999999999997e-06 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297169+0000", 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.032e-06 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.298950+0000", 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017802460000000001 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299039+0000", 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8962000000000006e-05 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:231378 5.5 e65/49 5:a3e915ed:::benchmark_data_smithi032_22800_object103264:head v 65'14298, mlcod=65'14293)", 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.297507+0000", 2024-04-18T22:24:43.324 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0020176170000000002, 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024212529999999999, 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297507+0000", 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297507+0000", 2024-04-18T22:24:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297508+0000", 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9699999999999995e-07 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297516+0000", 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5320000000000006e-06 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297524+0000", 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0199999999999994e-06 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297528+0000", 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4000000000000002e-06 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297571+0000", 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2287999999999999e-05 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.327 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297590+0000", 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9704e-05 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:231383 5.0 e65/61 5:00d37a6e:::benchmark_data_smithi032_22800_object103269:head v 65'14337, mlcod=65'14335)", 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.297610+0000", 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0019140400000000001, 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023754779999999999, 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297610+0000", 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297610+0000", 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297611+0000", 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.84e-07 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297617+0000", 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2670000000000003e-06 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297623+0000", 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4010000000000002e-06 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297628+0000", 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1490000000000002e-06 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297669+0000", 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0657000000000001e-05 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297688+0000", 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8640000000000001e-05 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.331 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231385 5.4 5:205ba16b:::benchmark_data_smithi032_22800_object103271:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.299208+0000", 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00031616100000000002, 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00083156599999999995, 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231385 2024-04-18T22:24:43.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299208+0000", 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299208+0000", 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.333 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299209+0000", 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6899999999999997e-07 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299213+0000", 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.109e-06 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299221+0000", 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0739999999999993e-06 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.334 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299227+0000", 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8359999999999997e-06 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299280+0000", 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2706e-05 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299415+0000", 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013501600000000001 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.335 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231386 5.3 5:cf6acecb:::benchmark_data_smithi032_22800_object103272:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.299240+0000", 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00028386600000000002, 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000870179, 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231386 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.336 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299240+0000", 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299240+0000", 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299241+0000", 2024-04-18T22:24:43.337 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8200000000000001e-07 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299242+0000", 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.731e-06 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299248+0000", 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1030000000000001e-06 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.338 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299251+0000", 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4929999999999998e-06 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299297+0000", 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5605000000000001e-05 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299373+0000", 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6364000000000003e-05 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.339 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299433+0000", 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9611000000000002e-05 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231387 5.7 5:e635c41e:::benchmark_data_smithi032_22800_object103273:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.299584+0000", 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999404, 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00059346400000000004, 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.340 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231387 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299584+0000", 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299584+0000", 2024-04-18T22:24:43.341 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299585+0000", 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8000000000000005e-07 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299589+0000", 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2520000000000001e-06 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.342 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299599+0000", 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0287e-05 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299604+0000", 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1959999999999999e-06 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299662+0000", 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7515000000000001e-05 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.343 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299791+0000", 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012883 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299907+0000", 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000116094 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231388 5.6 5:6cad9646:::benchmark_data_smithi032_22800_object103274:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.299623+0000", 2024-04-18T22:24:43.344 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9999013, 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061942300000000002, 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231388 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299623+0000", 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299623+0000", 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299623+0000", 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9599999999999999e-07 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299627+0000", 2024-04-18T22:24:43.346 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.162e-06 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299634+0000", 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9630000000000001e-06 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299637+0000", 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3799999999999998e-06 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299672+0000", 2024-04-18T22:24:43.347 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.5343999999999999e-05 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299753+0000", 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0789000000000002e-05 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.299814+0000", 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1257e-05 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.348 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231362 5.c 5:32a97781:::benchmark_data_smithi032_22800_object103248:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.288468+0000", 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.011056453000000001, 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0118402, 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231362 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.349 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288468+0000", 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288468+0000", 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.350 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288468+0000", 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9999999999999998e-07 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288471+0000", 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6970000000000002e-06 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288477+0000", 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2299999999999996e-06 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.351 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288483+0000", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5330000000000002e-06 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.288555+0000", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2637000000000003e-05 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.292870+0000", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043146440000000003 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.352 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293008+0000", 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013777900000000001 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.297289+0000", 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042810720000000003 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:231376 5.2 5:425373f9:::benchmark_data_smithi032_22800_object103262:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:43.293496+0000", 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0060281229999999998, 2024-04-18T22:24:43.353 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0068836879999999998, 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 231376 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293496+0000", 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293496+0000", 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293497+0000", 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7999999999999996e-07 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293499+0000", 2024-04-18T22:24:43.355 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6469999999999998e-06 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293505+0000", 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0700000000000003e-06 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293509+0000", 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7129999999999999e-06 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293556+0000", 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6455000000000002e-05 2024-04-18T22:24:43.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293704+0000", 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014838399999999999 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.293814+0000", 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000109916 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:43.296840+0000", 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030258329999999999 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.357 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.358 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.358 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:43.358 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 14 2024-04-18T22:24:43.358 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:43.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:43.360 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:43.359+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:43.460 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:43.459+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:43.552 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:43.552 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:43.552 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:43.552 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:43.552 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:43.561 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:43.557+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:43.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:43.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:43.657+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:43.763 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:43.763+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:43.770 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:43.770 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:43.770 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 40.733322631, 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.772 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.773 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.774 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:43.774 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:43.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.776 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.778 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.779 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:43.780 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 40.728679632000002, 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:43.781 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:43.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:43.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:43.785 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.786 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:43.787 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 40.719581894999997, 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.788 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.789 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.790 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.791 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:43.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:43.793 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:43.794 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 40.710561970000001, 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:43.796 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.797 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.798 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.799 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.800 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 40.694550602, 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:43.803 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:43.804 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:43.805 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.806 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 40.691747182, 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.808 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:43.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:43.812 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:43.813 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.814 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.882248218000001, 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.815 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.816 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.817 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.818 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:43.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.880582012, 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.821 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.822 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.824 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.879609516, 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.825 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:43.826 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:43.827 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.829 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.878249539, 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.830 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.831 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:43.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:43.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.875682408999999, 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.836 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.838 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.839 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.87462777, 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.842 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.843 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.844 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.845 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.846 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.873785829999999, 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:43.850 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:43.851 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.87197272, 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:43.852 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.853 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.854 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.871785668999999, 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.858 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.859 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.860 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.861 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.862 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.871546948000001, 2024-04-18T22:24:43.863 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.871357743000001, 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.871246206, 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:43.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:43.879 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.880 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.871069381, 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.881 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.883 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 12.868893656000001, 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.886 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:43.887 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:43.888 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:43.889 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:43.890 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:43.892 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:43.892 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:43.891+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:24:43.992 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:43.991+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:44.050 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:44.050 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:44.050 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.050 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:233237 5.5 5:a1c8b180:::benchmark_data_smithi032_22800_object105123:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T22:24:44.050 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.046804+0000", 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.003582333, 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036074129999999999, 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 233237 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.051 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046804+0000", 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046804+0000", 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046805+0000", 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.108e-06 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.052 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046810+0000", 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1939999999999997e-06 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046821+0000", 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1093e-05 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046828+0000", 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1300000000000003e-06 2024-04-18T22:24:44.053 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.047904+0000", 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0010752299999999999 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.047992+0000", 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8245000000000006e-05 2024-04-18T22:24:44.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048061+0000", 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8689000000000006e-05 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050336+0000", 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0022755679999999999 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.055 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:233235 5.3 e66/19 5:cffa22d9:::benchmark_data_smithi032_22800_object105121:head v 66'14870, mlcod=66'14867)", 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.048183+0000", 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0022033880000000001, 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0023194550000000002, 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048183+0000", 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048183+0000", 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048183+0000", 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0999999999999998e-07 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048189+0000", 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3070000000000002e-06 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.057 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048202+0000", 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2974e-05 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048208+0000", 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5189999999999998e-06 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048276+0000", 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8126000000000005e-05 2024-04-18T22:24:44.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048299+0000", 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.2555999999999999e-05 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:233232 5.c e66/19 5:37f66c61:::benchmark_data_smithi032_22800_object105118:head v 66'14521, mlcod=66'14517)", 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.048404+0000", 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0019823390000000001, 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021602599999999998, 2024-04-18T22:24:44.059 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048404+0000", 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048404+0000", 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.060 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048404+0000", 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9200000000000001e-07 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048409+0000", 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3680000000000004e-06 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048417+0000", 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1990000000000006e-06 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.061 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048423+0000", 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5289999999999997e-06 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048467+0000", 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3316000000000003e-05 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.048485+0000", 2024-04-18T22:24:44.062 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8621999999999999e-05 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:233240 5.2 e66/23 5:405f4ffe:::benchmark_data_smithi032_22800_object105126:head v 66'14635, mlcod=66'14633)", 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.049053+0000", 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0013336299999999999, 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0015711270000000001, 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.063 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049053+0000", 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049053+0000", 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049053+0000", 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6000000000000004e-07 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.064 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049061+0000", 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7549999999999994e-06 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049071+0000", 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0594000000000001e-05 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049078+0000", 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.426e-06 2024-04-18T22:24:44.065 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049128+0000", 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0149999999999999e-05 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049148+0000", 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9811999999999999e-05 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.066 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:233241 5.9 5:91e8634c:::benchmark_data_smithi032_22800_object105127:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.049212+0000", 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00117439, 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001448347, 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 233241 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.067 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049212+0000", 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049212+0000", 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049213+0000", 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2500000000000004e-07 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.068 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049218+0000", 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9099999999999996e-06 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049228+0000", 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0532e-05 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049237+0000", 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2080000000000006e-06 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.069 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049299+0000", 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1939000000000004e-05 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049476+0000", 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176434 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.070 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:233243 5.6 e66/19 5:668f389a:::benchmark_data_smithi032_22800_object105129:head v 66'14483, mlcod=66'14481)", 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.049635+0000", 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00075152300000000003, 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001060776, 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049635+0000", 2024-04-18T22:24:44.071 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.073 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.073 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.073 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049635+0000", 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049635+0000", 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3600000000000004e-07 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049641+0000", 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.575e-06 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.074 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049650+0000", 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2969999999999992e-06 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049657+0000", 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3669999999999999e-06 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049707+0000", 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9360999999999997e-05 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.075 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049728+0000", 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.075e-05 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:233239 5.3 e66/19 5:cbb2cae6:::benchmark_data_smithi032_22800_object105125:head v 66'14871, mlcod=66'14867)", 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.050077+0000", 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00030947, 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00064966000000000004, 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.076 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050077+0000", 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.077 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050077+0000", 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050077+0000", 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9399999999999995e-07 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050082+0000", 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7940000000000002e-06 2024-04-18T22:24:44.078 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050095+0000", 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2867000000000001e-05 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050103+0000", 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0309999999999998e-06 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050298+0000", 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000195255 2024-04-18T22:24:44.079 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050322+0000", 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3458000000000001e-05 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:233247 5.f 5:f5cdef0b:::benchmark_data_smithi032_22800_object105133:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.050328+0000", 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 5.8285999999999997e-05, 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00043174000000000001, 2024-04-18T22:24:44.080 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 233247 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050328+0000", 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.081 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050328+0000", 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050329+0000", 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2699999999999999e-07 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050333+0000", 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6120000000000001e-06 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.082 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050344+0000", 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0540000000000001e-05 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050350+0000", 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4860000000000002e-06 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050404+0000", 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3761000000000002e-05 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.083 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.050525+0000", 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012121999999999999 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:233230 5.1 5:85aa7cee:::benchmark_data_smithi032_22800_object105116:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:44.043969+0000", 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0064169880000000002, 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0068263159999999998, 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:44.084 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 233230 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.043969+0000", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.043969+0000", 2024-04-18T22:24:44.085 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.043970+0000", 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8999999999999995e-07 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.043975+0000", 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5210000000000004e-06 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.086 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.043984+0000", 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.798e-06 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.043992+0000", 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1100000000000003e-06 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046608+0000", 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026156550000000002 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.087 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046741+0000", 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013313800000000001 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.046858+0000", 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116289 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:44.049005+0000", 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0021473780000000001 2024-04-18T22:24:44.088 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.089 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:44.089 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.089 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:44.089 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:44.089 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:24:44.089 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:44.090 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:44.092 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:44.089+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:44.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:44.189+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:44.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:44.244 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:44.244 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:44.244 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:44.265 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:44.307 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:44.305+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:44.371 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:44.391 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:44.394 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:44.394 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:44.396 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:44.393+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:44.465 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:44.465 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:44.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:44.531 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:44.531+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:44.597 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:44.595+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:44.642 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:44.642 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:44.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:44.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2024-04-18T22:24:44.698 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:44.693+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:44.799 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:44.799+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:44.861 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:44.861 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:44.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:44.923 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:44.923+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:45.001 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:44.999+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:45.080 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:45.080 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:45.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:45.102 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.097+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.203 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.201+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.262 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:45.262 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:45.282 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:45.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.325+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.401+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.481 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:45.481 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:45.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:45.547 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.545+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.601+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.705 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:45.706 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:45.708 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.705+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:45.808 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:45.807+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:45.859 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:45.859 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:45.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:45.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:45.905+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:45.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:24:46.010 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:46.005+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:46.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:46.111 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:46.109+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:46.212 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:46.209+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:46.241 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:46.242 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:46.261 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:46.313 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.311+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:46.414 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.411+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.415 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:46.415 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:46.436 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:46.515 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.515+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.556 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:46.556 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:46.557 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:46.574 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:46.577 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:46.577 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:46.593 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:46.593 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:46.594 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:46.594 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:46.594 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:46.615 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:46.617 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.615+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.717 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.715+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.773 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.65749023399999, 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.774 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:46.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:46.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:46.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:46.778 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.657394388, 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.779 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.780 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.781 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.782 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.783 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.784 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.657337506, 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:46.785 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.786 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:46.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:46.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.789 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.65727783400001, 2024-04-18T22:24:46.790 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:46.791 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.792 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.793 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:46.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.65721987200001, 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.796 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:46.797 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.798 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.799 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.800 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.65700018000001, 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.801 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:46.802 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:46.803 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:46.804 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.805 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:46.806 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.656712559, 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.807 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.808 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.809 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.810 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.811 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.65656009099999, 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:46.812 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:46.813 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.814 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.815 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.816 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.654960512, 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:46.817 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.818 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.819 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.820 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.821 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 134.65486215999999, 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:46.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:46.824 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.825 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 100.973140207, 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:46.826 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:46.827 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 100.972485994, 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.830 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:46.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.832 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.833 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 100.972379299, 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.834 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:46.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.837 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 100.97051338999999, 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.839 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.840 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.841 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:46.842 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 100.969848067, 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.843 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.844 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:46.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.846 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:46.847 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 33.428032336000001, 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.848 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.849 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:46.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:46.851 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:46.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 33.421891838999997, 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.853 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:46.854 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:46.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.856 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.857 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:46.858 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.905585758000001, 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:46.860 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.861 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.862 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:46.863 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.904051834000001, 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:46.864 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:46.865 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:46.867 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:46.868 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 30.898226479000002, 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:46.869 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.870 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.871 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.872 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.873 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:46.874 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.875 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.875 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.875 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:46.875 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:46.875 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:46.876 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.875+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:46.977 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:46.975+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240632 5.a 5:5a1fb7a0:::benchmark_data_smithi032_22800_object112518:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.015629+0000", 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.005316126, 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053405309999999999, 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240632 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015629+0000", 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.022 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015629+0000", 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015631+0000", 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4479999999999999e-06 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015636+0000", 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2129999999999999e-06 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.023 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015648+0000", 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2669e-05 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015658+0000", 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3070000000000008e-06 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015721+0000", 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3439e-05 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.024 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015852+0000", 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000131261 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:240631 5.0 e67/61 5:01ba2096:::benchmark_data_smithi032_22800_object112517:head v 67'14901, mlcod=66'14897)", 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.015683+0000", 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.005261941, 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0053592049999999997, 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015683+0000", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015683+0000", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015684+0000", 2024-04-18T22:24:47.026 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4e-07 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015690+0000", 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5799999999999999e-06 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015699+0000", 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3009999999999995e-06 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015705+0000", 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3940000000000001e-06 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015752+0000", 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6486000000000002e-05 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.015778+0000", 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.6414000000000002e-05 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240636 5.6 5:69a672ef:::benchmark_data_smithi032_22800_object112522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.016223+0000", 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.004721823, 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048848930000000004, 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240636 2024-04-18T22:24:47.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016223+0000", 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016223+0000", 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.030 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016224+0000", 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.54e-07 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016234+0000", 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0054999999999999e-05 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016244+0000", 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.7089999999999997e-06 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.031 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016251+0000", 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7309999999999996e-06 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018258+0000", 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002007179 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018372+0000", 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011380800000000001 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.032 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018563+0000", 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019172899999999999 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240639 5.4 5:2d2f92c9:::benchmark_data_smithi032_22800_object112525:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.016311+0000", 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0046343859999999999, 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0048692099999999997, 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240639 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016311+0000", 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.034 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016311+0000", 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016311+0000", 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0299999999999999e-07 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016314+0000", 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.8320000000000002e-06 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.035 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016323+0000", 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3410000000000003e-06 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.016328+0000", 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5840000000000002e-06 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018046+0000", 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0017174079999999999 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.036 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018178+0000", 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013210699999999999 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240641 5.7 5:e4681024:::benchmark_data_smithi032_22800_object112527:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.018755+0000", 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021906410000000001, 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024954650000000001, 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.037 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240641 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018755+0000", 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018755+0000", 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018756+0000", 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4799999999999997e-07 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018760+0000", 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8520000000000003e-06 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.039 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018768+0000", 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018774+0000", 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8690000000000001e-06 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018825+0000", 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1269e-05 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.040 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018949+0000", 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012357199999999999 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019057+0000", 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000108278 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240642 5.b 5:db9bdee6:::benchmark_data_smithi032_22800_object112528:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.018792+0000", 2024-04-18T22:24:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021532140000000001, 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025288239999999998, 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240642 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018792+0000", 2024-04-18T22:24:47.042 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018792+0000", 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.043 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018793+0000", 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7899999999999999e-07 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018795+0000", 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.458e-06 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018801+0000", 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8860000000000001e-06 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.044 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018805+0000", 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4780000000000002e-06 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018868+0000", 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2946000000000003e-05 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020938+0000", 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020698629999999999 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240643 5.3 5:c6f2389c:::benchmark_data_smithi032_22800_object112529:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.018817+0000", 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0021280409999999998, 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002570705, 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240643 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018817+0000", 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018817+0000", 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018818+0000", 2024-04-18T22:24:47.047 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0400000000000002e-07 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018820+0000", 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1710000000000001e-06 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018826+0000", 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.511e-06 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.048 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.018831+0000", 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3189999999999996e-06 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019347+0000", 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00051612000000000001 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019489+0000", 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000142053 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.049 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019603+0000", 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011414 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:240645 5.1 e67/49 5:876e429c:::benchmark_data_smithi032_22800_object112531:head v 67'14967, mlcod=67'14964)", 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.019415+0000", 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001530726, 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.002047943, 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:47.050 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019415+0000", 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019415+0000", 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019415+0000", 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7000000000000005e-07 2024-04-18T22:24:47.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019420+0000", 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9699999999999998e-06 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019434+0000", 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4364e-05 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019440+0000", 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.908e-06 2024-04-18T22:24:47.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019496+0000", 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5234e-05 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019519+0000", 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3204999999999999e-05 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.053 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240646 5.8 5:1d7458d6:::benchmark_data_smithi032_22800_object112532:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.019908+0000", 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0010371919999999999, 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016008870000000001, 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240646 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.054 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019908+0000", 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019908+0000", 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019909+0000", 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6300000000000004e-07 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.055 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019913+0000", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7440000000000001e-06 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019923+0000", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0529e-05 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019930+0000", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4949999999999996e-06 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.056 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.019987+0000", 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7306000000000003e-05 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020255+0000", 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000267755 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:240647 5.7 5:e51c68ea:::benchmark_data_smithi032_22800_object112533:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:47.020424+0000", 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00052112600000000001, 2024-04-18T22:24:47.057 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.001130814, 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 240647 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020424+0000", 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020424+0000", 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020425+0000", 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8800000000000002e-07 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020428+0000", 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3230000000000002e-06 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.059 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020435+0000", 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8279999999999997e-06 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020439+0000", 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2289999999999997e-06 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020875+0000", 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00043606899999999997 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.060 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.020967+0000", 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.221e-05 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:47.021029+0000", 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1766999999999993e-05 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 10 2024-04-18T22:24:47.061 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:47.063 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:47.078 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.073+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:47.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.177+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:47.265 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:47.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:47.265 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:47.265 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:47.265 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:47.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.277+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:47.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:47.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.377+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:47.438 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 44.400785106000001, 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:47.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:47.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:47.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 44.396142107000003, 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:47.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:47.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 44.387044369999998, 2024-04-18T22:24:47.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:47.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:47.454 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.455 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.456 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.457 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 44.378024445000001, 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:47.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.459 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.460 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:47.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:47.462 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.463 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 44.362013077, 2024-04-18T22:24:47.464 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.467 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:47.470 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 44.359209657000001, 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.471 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:47.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:47.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:47.476 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.549710693000002, 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:47.477 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.478 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:47.479 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:47.480 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:47.481 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.482 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.548044486999999, 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.483 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:47.484 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:47.485 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.486 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.487 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.547071990999999, 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.488 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.489 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:47.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:47.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.492 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.545712013999999, 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.493 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:47.494 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.495 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.496 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.497 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.543144884, 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.499 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:47.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:47.501 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.502 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:47.503 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.542090245000001, 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:47.504 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:47.506 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:47.507 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:47.508 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:47.509 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.541248305, 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.510 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.539435194999999, 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.539248143999998, 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.523 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:47.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.539009423, 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:47.526 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.527 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.528 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.529 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.530 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.538820218000001, 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.531 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:47.532 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:47.533 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.534 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.535 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.538708680999999, 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.536 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:47.538 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.539 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.540 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.538531855999999, 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.541 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:47.543 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:47.544 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.545 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 16.536356131000002, 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:47.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:47.547 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.548 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:47.550 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:47.551 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:47.552 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:47.553 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:47.554 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:47.554 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:47.551+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:47.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.653+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:242371 5.1 5:8c0416b1:::benchmark_data_smithi032_22800_object114257:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:47.705269+0000", 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0053950059999999999, 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0054122700000000003, 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:47.710 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 242371 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705269+0000", 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:47.711 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705269+0000", 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705270+0000", 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5600000000000004e-07 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705274+0000", 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.391e-06 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:47.712 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705285+0000", 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.075e-05 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705292+0000", 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0400000000000004e-06 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705337+0000", 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5003000000000003e-05 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.713 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705454+0000", 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011702 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705564+0000", 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000109664 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709025+0000", 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003461567 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:242372 5.1 5:80ff98c5:::benchmark_data_smithi032_22800_object114258:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:47.705307+0000", 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.005356347, 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0054227449999999996, 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 242372 2024-04-18T22:24:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705307+0000", 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705307+0000", 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705308+0000", 2024-04-18T22:24:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.08e-07 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705310+0000", 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.3929999999999998e-06 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705317+0000", 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3629999999999999e-06 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705323+0000", 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.3219999999999998e-06 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707259+0000", 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0019356009999999999 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707389+0000", 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000130409 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707504+0000", 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011488 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.710183+0000", 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0026796120000000001 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:242374 5.5 5:adcafd14:::benchmark_data_smithi032_22800_object114260:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:47.705355+0000", 2024-04-18T22:24:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0053092349999999998, 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0054211859999999997, 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 242374 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705355+0000", 2024-04-18T22:24:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705355+0000", 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705355+0000", 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6100000000000001e-07 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705357+0000", 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0880000000000002e-06 2024-04-18T22:24:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705363+0000", 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0260000000000004e-06 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705368+0000", 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.493e-06 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705427+0000", 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9719999999999997e-05 2024-04-18T22:24:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705543+0000", 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115357 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.705649+0000", 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106674 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707849+0000", 2024-04-18T22:24:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.002199126 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:242377 5.0 5:02fbfacf:::benchmark_data_smithi032_22800_object114263:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:47.707590+0000", 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0030740540000000001, 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0032309700000000001, 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 242377 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707590+0000", 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707590+0000", 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707591+0000", 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.274e-06 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707597+0000", 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.8329999999999999e-06 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707606+0000", 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0070000000000003e-06 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.726 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707611+0000", 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707666+0000", 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5347999999999999e-05 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707793+0000", 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012602199999999999 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.727 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.707904+0000", 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111278 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709767+0000", 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018632029999999999 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:242381 5.1 5:832069a0:::benchmark_data_smithi032_22800_object114267:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:47.708955+0000", 2024-04-18T22:24:47.728 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0017084279999999999, 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001906102, 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 242381 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.708955+0000", 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.708955+0000", 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.708956+0000", 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4799999999999997e-07 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.708961+0000", 2024-04-18T22:24:47.730 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5229999999999997e-06 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.708973+0000", 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.172e-05 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.708980+0000", 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0269999999999998e-06 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709048+0000", 2024-04-18T22:24:47.731 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7990000000000005e-05 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709163+0000", 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115689 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709235+0000", 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1878000000000005e-05 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.710403+0000", 2024-04-18T22:24:47.732 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011676080000000001 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:242380 5.6 e67/19 5:633c623e:::benchmark_data_smithi032_22800_object114266:head v 67'15029, mlcod=67'15027)", 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:47.709074+0000", 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0015901649999999999, 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001829074, 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.733 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709074+0000", 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709074+0000", 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709074+0000", 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.13e-07 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.734 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709079+0000", 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6120000000000001e-06 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709088+0000", 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4849999999999998e-06 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709094+0000", 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1190000000000002e-06 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.735 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709147+0000", 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2555000000000001e-05 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:47.709167+0000", 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9678e-05 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 6 2024-04-18T22:24:47.736 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:47.738 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:47.755 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:47.755+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:47.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.853+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:47.930 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:47.931 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:47.931 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:47.931 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:47.931 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:47.950 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:47.995 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:47.993+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:48.059 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.059+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02724737599999, 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.144 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:48.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.146 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.147 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.148 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.149 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02715153, 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.150 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.151 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.152 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:48.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:48.154 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.027094648, 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.155 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.156 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:48.157 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.158 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.159 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:48.160 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02703497600001, 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:48.162 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:48.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.163 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.163 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.163 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:48.165 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:48.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.165 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:48.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.167 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02697701400001, 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.168 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.169 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.170 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.171 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.172 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.173 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02675732200001, 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:48.174 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:48.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.178 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.179 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:48.180 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.026469701, 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:48.181 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.182 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.183 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:48.184 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:48.185 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.186 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02631723299999, 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.187 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.188 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:48.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.190 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:48.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.024717654, 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:48.192 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.194 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.195 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:48.196 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 136.02461930199999, 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:48.198 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:48.199 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:48.200 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:48.201 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 102.342897349, 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.202 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:48.203 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:48.204 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:48.205 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:48.206 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 102.34224313599999, 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:48.207 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.208 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.209 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.210 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 102.34213644099999, 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:48.211 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.212 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.214 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 102.34027053200001, 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.216 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.217 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:48.218 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.219 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 102.339605209, 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.220 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:48.221 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:48.222 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.223 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.797789477999999, 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.224 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.226 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.227 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:48.228 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:48.229 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 34.791648981000002, 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:48.230 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.231 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.232 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.233 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:48.234 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.235 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.275342899999998, 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.236 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.237 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:48.238 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:48.239 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:48.240 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.273808975999998, 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:48.241 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.242 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:48.244 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.245 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.246 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 32.267983620999999, 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:48.247 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:48.248 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:48.249 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:48.250 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.252 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:48.253 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:48.253 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:48.254 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.251+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:48.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:48.355 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.355+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:48.406 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244046 5.2 5:4faee962:::benchmark_data_smithi032_22800_object115932:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.400761+0000", 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.005962162, 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0059671560000000004, 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.407 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244046 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400761+0000", 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400761+0000", 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.408 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400761+0000", 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7599999999999997e-07 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400764+0000", 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5320000000000001e-06 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400770+0000", 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7490000000000001e-06 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.409 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400776+0000", 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6219999999999997e-06 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400880+0000", 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000103844 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400955+0000", 2024-04-18T22:24:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5147000000000004e-05 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401016+0000", 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0708999999999998e-05 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.405468+0000", 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0044519039999999996 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406683+0000", 2024-04-18T22:24:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00121535 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406708+0000", 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5262e-05 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406728+0000", 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9561000000000001e-05 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244052 5.b 5:db1d6bde:::benchmark_data_smithi032_22800_object115938:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.401065+0000", 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0056576459999999997, 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057368039999999999, 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.413 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244052 2024-04-18T22:24:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.414 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401065+0000", 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401065+0000", 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.415 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401066+0000", 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5199999999999995e-07 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401071+0000", 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.673e-06 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401079+0000", 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4479999999999994e-06 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401086+0000", 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6329999999999999e-06 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.405423+0000", 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043377169999999996 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.405557+0000", 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013309300000000001 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.417 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244056 5.3 5:c8d995c8:::benchmark_data_smithi032_22800_object115942:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.402175+0000", 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0045480659999999999, 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046627329999999996, 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244056 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.418 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402175+0000", 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402175+0000", 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402176+0000", 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4499999999999995e-07 2024-04-18T22:24:48.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402181+0000", 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1619999999999999e-06 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402193+0000", 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1759e-05 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402201+0000", 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7609999999999996e-06 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.420 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.405409+0000", 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0032078950000000001 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.405528+0000", 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000118433 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.405639+0000", 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000111488 2024-04-18T22:24:48.421 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:244049 5.0 e67/61 5:0d7edd0e:::benchmark_data_smithi032_22800_object115935:head v 67'15101, mlcod=67'15098)", 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.402868+0000", 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.003854335, 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.004006941, 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.422 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402868+0000", 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402868+0000", 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402869+0000", 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3600000000000003e-07 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.423 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402878+0000", 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6559999999999998e-06 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402887+0000", 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2359999999999993e-06 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402893+0000", 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.105e-06 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.424 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402925+0000", 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.1872999999999998e-05 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402937+0000", 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.1909000000000001e-05 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244058 5.6 5:6fbf7999:::benchmark_data_smithi032_22800_object115944:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.404100+0000", 2024-04-18T22:24:48.425 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0026225649999999999, 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0028055749999999998, 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244058 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404100+0000", 2024-04-18T22:24:48.426 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404100+0000", 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404101+0000", 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1699999999999997e-07 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404105+0000", 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6849999999999999e-06 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404114+0000", 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4390000000000006e-06 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404119+0000", 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7870000000000001e-06 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.428 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404157+0000", 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8334999999999998e-05 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404231+0000", 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4071000000000003e-05 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404294+0000", 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3078999999999997e-05 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244059 5.3 5.255cbb83 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.404252+0000", 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00247094, 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0026891549999999999, 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244059 2024-04-18T22:24:48.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404252+0000", 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404252+0000", 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404252+0000", 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9900000000000001e-07 2024-04-18T22:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404255+0000", 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.531e-06 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404260+0000", 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404264+0000", 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7160000000000001e-06 2024-04-18T22:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:244051 5.0 e67/61 5:055f25b7:::benchmark_data_smithi032_22800_object115937:head v 67'15102, mlcod=67'15098)", 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.404510+0000", 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0022127359999999999, 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0024583510000000001, 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.433 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404510+0000", 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404510+0000", 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404511+0000", 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6800000000000001e-07 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.434 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404514+0000", 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8920000000000002e-06 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404522+0000", 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.362e-06 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404526+0000", 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1690000000000002e-06 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.435 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404568+0000", 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2287999999999999e-05 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.404583+0000", 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4754e-05 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244060 5.3 5.989fdbc3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.406145+0000", 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00057811199999999998, 2024-04-18T22:24:48.436 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00086051699999999999, 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244060 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406145+0000", 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406145+0000", 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406145+0000", 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9100000000000003e-07 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406148+0000", 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9490000000000001e-06 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406158+0000", 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.3300000000000005e-06 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.406163+0000", 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.921e-06 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:244062 5.1 e67/49)", 2024-04-18T22:24:48.439 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.407007+0000", 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997153, 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.9774999999999997e-05, 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.407007+0000", 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.407007+0000", 2024-04-18T22:24:48.440 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.407008+0000", 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5199999999999997e-07 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.407012+0000", 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4889999999999997e-06 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.441 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.407022+0000", 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0059e-05 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.407029+0000", 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.297e-06 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244040 5.b 5:d036d341:::benchmark_data_smithi032_22800_object115926:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.397727+0000", 2024-04-18T22:24:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0089955769999999994, 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093615250000000007, 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244040 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397727+0000", 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397727+0000", 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397728+0000", 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8400000000000004e-07 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397732+0000", 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8920000000000002e-06 2024-04-18T22:24:48.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397740+0000", 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5990000000000002e-06 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397746+0000", 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9959999999999999e-06 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.399559+0000", 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0018125459999999999 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.399677+0000", 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011854999999999999 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244042 5.b 5:d2bf62bc:::benchmark_data_smithi032_22800_object115928:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.397794+0000", 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0089283990000000001, 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0093291950000000002, 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244042 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397794+0000", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397794+0000", 2024-04-18T22:24:48.447 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397795+0000", 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1600000000000002e-07 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397797+0000", 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4200000000000001e-06 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397803+0000", 2024-04-18T22:24:48.448 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.1670000000000004e-06 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.397808+0000", 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7520000000000004e-06 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.401682+0000", 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0038741359999999998 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.402007+0000", 2024-04-18T22:24:48.449 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032471300000000002 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:244045 5.b 5:d496cab2:::benchmark_data_smithi032_22800_object115931:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:48.400724+0000", 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0059989249999999996, 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0064338290000000003, 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:48.450 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 244045 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400724+0000", 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400724+0000", 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.451 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400725+0000", 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3489999999999999e-06 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400730+0000", 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0499999999999999e-06 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400740+0000", 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.978e-06 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.452 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.400748+0000", 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4449999999999999e-06 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.403652+0000", 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029042370000000001 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:48.403832+0000", 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017974000000000001 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:48.453 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:48.454 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 12 2024-04-18T22:24:48.454 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:48.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:48.456 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:48.453+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:48.557 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.555+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:48.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:48.650 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:48.650 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:48.650 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:48.658 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.655+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:48.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:48.738 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:48.758 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:48.759 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.759+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:48.761 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:48.762 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:48.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:48.857+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:48.866 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:48.866 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:48.886 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:48.961 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:48.959+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:49.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:24:49.041 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:49.041 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:49.062 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:49.062 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:49.057+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:49.162 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:49.159+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:49.254 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:49.255 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:49.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:49.261+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:49.273 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:49.364 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:49.361+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:49.464 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:49.463+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:49.468 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:49.468 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:49.488 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:49.565 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:49.561+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:49.632 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:49.633 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:49.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:49.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:49.661+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:49.767 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:49.767+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:49.818 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:49.907 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:49.907+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:49.968 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:49.965+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:24:50.008 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:50.009 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:50.028 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:50.075 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.073+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.170 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.165+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.184 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:50.184 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:50.204 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:50.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.289+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.372 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:50.371+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:50.398 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:50.399 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:50.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:50.504 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~76s 2024-04-18T22:24:50.511 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.509+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.573 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.569+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.609 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:50.609 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:50.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:50.675 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.673+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T22:24:50.775 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:50.775+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:50.780 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:50.780 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:50.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:50.887 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:50.887+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:50.925 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:50.944 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:50.947 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:50.947 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:50.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:50.977+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:50.993 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:50.993 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:50.993 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:51.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:51.099 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:51.099+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:51.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:51.177+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.172664310000002, 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:51.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:51.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.168021310999997, 2024-04-18T22:24:51.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:51.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:51.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.158923573999999, 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.226 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.230 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.149903649000002, 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:51.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.133892281000001, 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.238 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:51.240 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:51.241 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.244 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 48.131088861000002, 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.245 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:51.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.247 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.248 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:51.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.250 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.321589896999999, 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.251 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.252 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.255 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.319923691, 2024-04-18T22:24:51.256 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.257 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:51.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.260 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.318951195, 2024-04-18T22:24:51.261 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:51.263 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:51.264 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.265 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.266 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.317591218, 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.267 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:51.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:51.269 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.270 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.272 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.315024088000001, 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.273 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:51.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.275 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.276 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.277 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.313969448999998, 2024-04-18T22:24:51.278 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:51.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:51.281 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:51.282 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:51.283 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.313127509000001, 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.284 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:51.286 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.287 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.288 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:51.289 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.311314399, 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.290 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.291 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.292 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.293 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:51.294 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.311127347999999, 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.296 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.299 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.300 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.310888627000001, 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:51.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.302 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:51.305 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.310699421999999, 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.306 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.308 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.309 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:51.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.310587885, 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:51.311 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.312 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:51.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:51.315 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.31041106, 2024-04-18T22:24:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.319 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.320 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:51.321 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 20.308235334999999, 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:51.322 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:51.323 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.324 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.325 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:51.326 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:51.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:51.328 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:51.327+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:51.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:51.425+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:51.484 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:51.484 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:51.484 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.484 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251454 5.d 5:be8eb42f:::benchmark_data_smithi032_22800_object123340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.479161+0000", 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0059168759999999997, 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059418559999999997, 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251454 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.485 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479161+0000", 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479161+0000", 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479161+0000", 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5999999999999999e-07 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.486 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479164+0000", 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.6749999999999998e-06 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479174+0000", 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7999999999999993e-06 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479180+0000", 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1759999999999998e-06 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.487 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479231+0000", 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0946000000000003e-05 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479381+0000", 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015007700000000001 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251456 5.d 5:b3fcb5a9:::benchmark_data_smithi032_22800_object123342:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.488 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.479191+0000", 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0058867490000000001, 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0059888179999999999, 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251456 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479191+0000", 2024-04-18T22:24:51.489 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479191+0000", 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479192+0000", 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9399999999999995e-07 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.490 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479194+0000", 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1909999999999999e-06 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479199+0000", 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.5879999999999997e-06 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.479204+0000", 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7010000000000004e-06 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.491 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483295+0000", 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040911610000000003 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483407+0000", 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011199700000000001 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251460 5.5 5:aff18c20:::benchmark_data_smithi032_22800_object123346:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.481773+0000", 2024-04-18T22:24:51.492 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033049989999999999, 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.003479252, 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251460 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.481773+0000", 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.481773+0000", 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.481774+0000", 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.494 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.481778+0000", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9990000000000002e-06 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.481786+0000", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.8300000000000002e-06 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.481791+0000", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7380000000000002e-06 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483285+0000", 2024-04-18T22:24:51.495 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0014935479999999999 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483443+0000", 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015816100000000001 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483565+0000", 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000122655 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251465 5.d 5:bc1aeba2:::benchmark_data_smithi032_22800_object123351:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.482557+0000", 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.002520301, 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0027414480000000001, 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251465 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.497 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.482557+0000", 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.482557+0000", 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.482558+0000", 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6700000000000003e-07 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.498 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.482561+0000", 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7310000000000002e-06 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.482569+0000", 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3240000000000003e-06 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.482573+0000", 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4120000000000003e-06 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.499 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484531+0000", 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001957042 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484604+0000", 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3576000000000002e-05 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251468 5.f 5:f0c8ba45:::benchmark_data_smithi032_22800_object123354:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.500 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.483448+0000", 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001629346, 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001887351, 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251468 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483448+0000", 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.501 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483448+0000", 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483449+0000", 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1699999999999997e-07 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483456+0000", 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2269999999999996e-06 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.502 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483466+0000", 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0125999999999999e-05 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483474+0000", 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0129999999999993e-06 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483521+0000", 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6668000000000001e-05 2024-04-18T22:24:51.503 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483612+0000", 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.0400000000000002e-05 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251470 5.d 5.b185118d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.483632+0000", 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0014461400000000001, 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0017373219999999999, 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:51.504 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251470 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483632+0000", 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483632+0000", 2024-04-18T22:24:51.505 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483632+0000", 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.23e-07 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483637+0000", 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.481e-06 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483647+0000", 2024-04-18T22:24:51.506 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6819999999999998e-06 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.483653+0000", 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:251463 5.c e68/19 5:370a5386:::benchmark_data_smithi032_22800_object123349:head v 68'15719, mlcod=68'15717)", 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.484097+0000", 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00098074100000000003, 2024-04-18T22:24:51.507 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001302662, 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484097+0000", 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484097+0000", 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484098+0000", 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8699999999999995e-07 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484104+0000", 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8839999999999997e-06 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.509 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484113+0000", 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9719999999999998e-06 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484120+0000", 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9210000000000003e-06 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484172+0000", 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2074999999999999e-05 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484192+0000", 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9780999999999999e-05 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:251471 5.d 5.233448ed (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:51.484243+0000", 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00083433300000000004, 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0011884039999999999, 2024-04-18T22:24:51.511 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 251471 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484243+0000", 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.512 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484243+0000", 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484244+0000", 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8400000000000003e-07 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484251+0000", 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.2979999999999997e-06 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.513 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484260+0000", 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6260000000000007e-06 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:51.484266+0000", 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0689999999999998e-06 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.514 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:51.515 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 8 2024-04-18T22:24:51.515 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:51.516 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:51.563 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:51.561+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:51.630 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:51.627+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:51.712 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:51.712 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [], 2024-04-18T22:24:51.712 INFO:teuthology.orchestra.run.smithi196.stdout: "complaint_time": 30, 2024-04-18T22:24:51.713 INFO:teuthology.orchestra.run.smithi196.stdout: "num_blocked_ops": 0 2024-04-18T22:24:51.713 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:51.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:51.729+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:51.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:51.833 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:51.831+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout: "size": 20, 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 600, 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:8 3.10 3:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.931 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81535100900001, 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.033477324000000003, 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115138+0000", 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115139+0000", 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3430000000000001e-06 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115177+0000", 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7453999999999997e-05 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115197+0000", 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0058999999999999e-05 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115212+0000", 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5565000000000001e-05 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.124617+0000", 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0094042970000000007 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.934 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125592+0000", 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00097507900000000001 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.125915+0000", 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00032296700000000001 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137955+0000", 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012040331 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148352+0000", 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010397172999999999 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148504+0000", 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.148615+0000", 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111784 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.936 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:9 3.10 3:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81525516299999, 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036422293000000001, 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:51.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115234+0000", 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115235+0000", 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3999999999999999e-06 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.938 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115243+0000", 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4159999999999998e-06 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115260+0000", 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7414999999999998e-05 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115274+0000", 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3816e-05 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.939 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126573+0000", 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011299415 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.126800+0000", 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00022669500000000001 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127110+0000", 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00031001599999999998 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.940 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137969+0000", 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.010859120999999999 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151397+0000", 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013427902 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151548+0000", 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015127200000000001 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.941 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151656+0000", 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010782500000000001 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:10 3.10 3:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81519828099999, 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036793996000000002, 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.942 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115291+0000", 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115292+0000", 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:51.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115299+0000", 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0530000000000001e-06 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115317+0000", 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7861000000000001e-05 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115330+0000", 2024-04-18T22:24:51.944 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3213e-05 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130016+0000", 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.014685965 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130541+0000", 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00052535500000000003 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130719+0000", 2024-04-18T22:24:51.945 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000177889 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.138001+0000", 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0072821200000000004 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151815+0000", 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.013814267999999999 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.946 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151979+0000", 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016305399999999999 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152085+0000", 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010618799999999999 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:11 3.10 3:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.815138609, 2024-04-18T22:24:51.947 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037030971000000003, 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115350+0000", 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115351+0000", 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.11e-06 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115358+0000", 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8029999999999999e-06 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115375+0000", 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6748999999999998e-05 2024-04-18T22:24:51.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115391+0000", 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5594999999999999e-05 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131141+0000", 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015750443999999999 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131353+0000", 2024-04-18T22:24:51.950 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00021236000000000001 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131688+0000", 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00033442600000000003 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140084+0000", 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0083964450000000006 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152183+0000", 2024-04-18T22:24:51.951 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012098271000000001 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152265+0000", 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2810000000000002e-05 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152381+0000", 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000115958 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.952 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:12 3.10 3:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.815080647, 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037992933999999999, 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.953 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115408+0000", 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115409+0000", 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1230000000000001e-06 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115416+0000", 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.336e-06 2024-04-18T22:24:51.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115436+0000", 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9908999999999999e-05 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115453+0000", 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6855e-05 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.132977+0000", 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.017524319999999999 2024-04-18T22:24:51.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133274+0000", 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000297113 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.133679+0000", 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040543099999999999 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.140439+0000", 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0067594170000000002 2024-04-18T22:24:51.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153093+0000", 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012654121000000001 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153256+0000", 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00016348400000000001 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153401+0000", 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00014482500000000001 2024-04-18T22:24:51.957 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:25 3.10 3:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.814860955, 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.038301469999999997, 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.958 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115628+0000", 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115629+0000", 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.054e-06 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.959 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115636+0000", 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.7129999999999999e-06 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115653+0000", 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7014999999999999e-05 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115666+0000", 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3628999999999999e-05 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.960 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.136848+0000", 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.021181467999999998 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137066+0000", 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000217846 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137334+0000", 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00026810799999999997 2024-04-18T22:24:51.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.145228+0000", 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0078937079999999993 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153728+0000", 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0085003849999999992 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153823+0000", 2024-04-18T22:24:51.962 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5192000000000006e-05 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153930+0000", 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000106352 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:1 3.14 3:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81457333399999, 2024-04-18T22:24:51.963 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.036941884000000001, 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115916+0000", 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115917+0000", 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4600000000000003e-07 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115925+0000", 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.9500000000000001e-06 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115942+0000", 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7328000000000001e-05 2024-04-18T22:24:51.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.115956+0000", 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.4452e-05 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127020+0000", 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.011063524999999999 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.127379+0000", 2024-04-18T22:24:51.966 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000359561 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.128184+0000", 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00080471600000000003 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137165+0000", 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0089812280000000008 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152486+0000", 2024-04-18T22:24:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.015321111 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152730+0000", 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024357099999999999 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.152858+0000", 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000127496 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.968 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:3 3.14 3:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81442086600001, 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.037298498999999999, 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.969 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116068+0000", 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116069+0000", 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2759999999999999e-06 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116088+0000", 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8382000000000001e-05 2024-04-18T22:24:51.970 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116107+0000", 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9474999999999999e-05 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.116126+0000", 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8722000000000001e-05 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129031+0000", 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012904854 2024-04-18T22:24:51.971 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129402+0000", 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00037102100000000002 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.129647+0000", 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00024465400000000002 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.137303+0000", 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0076569400000000001 2024-04-18T22:24:51.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153132+0000", 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01582809 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153308+0000", 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000176033 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153367+0000", 2024-04-18T22:24:51.973 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9052000000000003e-05 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:23 3.d 3:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81282128699999, 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.034036213000000003, 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.974 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117668+0000", 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117669+0000", 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.7799999999999999e-06 2024-04-18T22:24:51.975 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117681+0000", 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1099e-05 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117706+0000", 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5655000000000001e-05 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117725+0000", 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.9108000000000001e-05 2024-04-18T22:24:51.976 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121073+0000", 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0033476360000000002 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121252+0000", 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00017882900000000001 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 1", 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.121506+0000", 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00025417000000000001 2024-04-18T22:24:51.977 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.131972+0000", 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01046565 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151523+0000", 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.01955088 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151598+0000", 2024-04-18T22:24:51.978 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.5056999999999996e-05 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.151704+0000", 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010634899999999999 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(mds.0.3:24 3.14 3:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e17)", 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 139.81272293500001, 2024-04-18T22:24:51.979 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.035870667000000002, 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117766+0000", 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.980 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117768+0000", 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5400000000000001e-06 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117774+0000", 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5690000000000001e-06 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117795+0000", 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1039e-05 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.981 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.117811+0000", 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5735000000000001e-05 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130628+0000", 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.012816721999999999 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.130927+0000", 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00029892599999999998 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:22:32.153637+0000", 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.022710135999999999 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25655 5.1 e26/23 5:8e09d0a0:::benchmark_data_smithi032_22800_object25654:head v 26'1541, mlcod=26'1538)", 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 106.131000982, 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.088140399999999994, 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.983 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799488+0000", 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.984 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799489+0000", 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0300000000000001e-06 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799539+0000", 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9815000000000003e-05 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799549+0000", 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.8959999999999998e-06 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.985 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799556+0000", 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6190000000000002e-06 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799610+0000", 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.3560999999999998e-05 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.799626+0000", 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6379999999999999e-05 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.986 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800784+0000", 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001157588 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887628+0000", 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.086844510999999999 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25656 5.c e26/19 5:32839d89:::benchmark_data_smithi032_22800_object25655:head v 26'1643, mlcod=26'1641)", 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:51.987 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 106.130346769, 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090817397999999994, 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800142+0000", 2024-04-18T22:24:51.988 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800143+0000", 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0900000000000001e-07 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800194+0000", 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0994e-05 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800202+0000", 2024-04-18T22:24:51.989 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.2910000000000007e-06 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800208+0000", 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.75e-06 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800319+0000", 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000111511 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.990 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800359+0000", 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.9962999999999997e-05 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890849+0000", 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090489395 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890960+0000", 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011088500000000001 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.991 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25657 5.6 e26/19 5:617f461b:::benchmark_data_smithi032_22800_object25656:head v 26'1585, mlcod=26'1582)", 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 106.130240074, 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090744530000000004, 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.992 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800249+0000", 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800250+0000", 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.82e-07 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800301+0000", 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1078999999999997e-05 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.993 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800311+0000", 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0180000000000001e-05 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800316+0000", 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.7829999999999998e-06 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800375+0000", 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.9101000000000002e-05 2024-04-18T22:24:51.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.800391+0000", 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.5973000000000001e-05 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890898+0000", 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090507200999999995 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890993+0000", 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5631000000000006e-05 2024-04-18T22:24:51.995 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25660 5.5 e26/19 5:a969ea30:::benchmark_data_smithi032_22800_object25659:head v 26'1535, mlcod=26'1533)", 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 106.128374165, 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.089009224999999997, 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:51.996 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802115+0000", 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802116+0000", 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.0599999999999999e-07 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.997 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802176+0000", 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0432e-05 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886787+0000", 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084611101999999994 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886812+0000", 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.4453000000000001e-05 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.998 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886968+0000", 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000156415 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887018+0000", 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0365000000000003e-05 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.890976+0000", 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0039580769999999999 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:51.999 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.891124+0000", 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000147575 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:25664 5.f e26/23 5:f9eab9f2:::benchmark_data_smithi032_22800_object25663:head v 26'1529, mlcod=26'1527)", 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 106.127708842, 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.090853984999999998, 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.000 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802780+0000", 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.001 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802781+0000", 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.7000000000000003e-07 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.802882+0000", 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000100548 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886900+0000", 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.084018232999999998 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.002 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.886925+0000", 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.5415999999999999e-05 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887311+0000", 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00038546700000000001 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.887359+0000", 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8025000000000002e-05 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.003 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893521+0000", 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0061624870000000003 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:23:05.893634+0000", 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00011283899999999999 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.004 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161660 5.1 5:8258ee96:::benchmark_data_smithi032_22800_object33546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.585893110999997, 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032794434999999997, 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161660 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.005 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344596+0000", 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344597+0000", 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.249e-06 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.006 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344605+0000", 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1829999999999994e-06 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344617+0000", 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1275e-05 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344625+0000", 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3909999999999998e-06 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.007 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344690+0000", 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5353999999999998e-05 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344818+0000", 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012778299999999999 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.344946+0000", 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012756200000000001 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.008 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.347045+0000", 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0020990090000000002 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.376933+0000", 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.029888416000000001 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377083+0000", 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000150023 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.009 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377390+0000", 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00030718999999999999 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:161676 5.1 5:865ff486:::benchmark_data_smithi032_22800_object33562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 38.579752614, 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.032897199000000002, 2024-04-18T22:24:52.010 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 161676 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.011 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350736+0000", 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350738+0000", 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.2470000000000001e-06 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350743+0000", 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4829999999999999e-06 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.012 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350752+0000", 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.2289999999999992e-06 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350760+0000", 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.4379999999999998e-06 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350816+0000", 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.6407999999999999e-05 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.013 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.350933+0000", 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116474 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.351016+0000", 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.3325999999999995e-05 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.377644+0000", 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.026628269999999999 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383461+0000", 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0058165600000000001 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383521+0000", 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.0007000000000003e-05 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:13.383634+0000", 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000112757 2024-04-18T22:24:52.015 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167491 5.1 5:8a2b4933:::benchmark_data_smithi032_22800_object39377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 36.063446532999997, 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.069377123999999998, 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.016 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167491 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867042+0000", 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.017 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867043+0000", 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9100000000000004e-07 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867046+0000", 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.7350000000000001e-06 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867053+0000", 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8650000000000003e-06 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.018 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867058+0000", 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0270000000000003e-06 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867098+0000", 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.0642999999999997e-05 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867214+0000", 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000116147 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.019 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.867311+0000", 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.6341000000000005e-05 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.873973+0000", 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0066626039999999999 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936226+0000", 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062252975000000002 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.020 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936295+0000", 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.8825000000000006e-05 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936420+0000", 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012447099999999999 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167495 5.5 5:a2490e08:::benchmark_data_smithi032_22800_object39381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 36.061912608999997, 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.071675753999999994, 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167495 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:52.022 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868576+0000", 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868577+0000", 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1699999999999997e-07 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868583+0000", 2024-04-18T22:24:52.023 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4269999999999999e-06 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868593+0000", 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.0359e-05 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.868600+0000", 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.221e-06 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.024 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869027+0000", 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00042683 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869125+0000", 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.781e-05 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.869190+0000", 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.4734999999999996e-05 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:52.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874156+0000", 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0049665350000000002 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940109+0000", 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.065952730000000001 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940157+0000", 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8547000000000001e-05 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.940252+0000", 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.4642999999999995e-05 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:167503 5.1 5:88805556:::benchmark_data_smithi032_22800_object39389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 36.056087253999998, 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.062384233999999997, 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.027 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 167503 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874402+0000", 2024-04-18T22:24:52.028 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874403+0000", 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1200000000000001e-07 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874407+0000", 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.1980000000000003e-06 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874416+0000", 2024-04-18T22:24:52.029 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.9579999999999996e-06 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874424+0000", 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.8990000000000001e-06 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874461+0000", 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7697999999999997e-05 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.030 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874601+0000", 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00013990300000000001 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.874755+0000", 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015327599999999999 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "op_commit", 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.879357+0000", 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0046021940000000004 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.031 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936558+0000", 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.057201250000000002 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "commit_sent", 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936630+0000", 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2291000000000003e-05 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "done", 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:15.936786+0000", 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00015565500000000001 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.032 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.033 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.033 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.033 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.033 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:52.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T22:24:52.035 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:52.029+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:52.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:52.134 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:52.135+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253180 5.6 5:65326b29:::benchmark_data_smithi032_22800_object125066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.184721+0000", 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0029124939999999998, 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0029397830000000001, 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253180 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184721+0000", 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.189 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184721+0000", 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184722+0000", 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.2799999999999995e-07 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184725+0000", 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7239999999999998e-06 2024-04-18T22:24:52.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184735+0000", 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8600000000000005e-06 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184741+0000", 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6400000000000002e-06 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185451+0000", 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00071002000000000001 2024-04-18T22:24:52.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185552+0000", 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000101186 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185645+0000", 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.2954000000000005e-05 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186865+0000", 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0012196970000000001 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.192 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253181 5.c 5:3c0d537c:::benchmark_data_smithi032_22800_object125067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.184751+0000", 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0028820170000000002, 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0030200639999999998, 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253181 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184751+0000", 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184751+0000", 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.194 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184752+0000", 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.84e-07 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184754+0000", 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2869999999999999e-06 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184760+0000", 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9220000000000002e-06 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.195 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.184765+0000", 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6110000000000004e-06 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185214+0000", 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00044908699999999998 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185279+0000", 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5296999999999995e-05 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.196 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185353+0000", 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3699e-05 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187403+0000", 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020505179999999999 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253182 5.3 5:cf2cff49:::benchmark_data_smithi032_22800_object125068:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.185280+0000", 2024-04-18T22:24:52.197 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0023531450000000001, 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0025909219999999998, 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253182 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185280+0000", 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185280+0000", 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185281+0000", 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3900000000000004e-07 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185285+0000", 2024-04-18T22:24:52.199 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.563e-06 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185294+0000", 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.1290000000000002e-06 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185300+0000", 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.998e-06 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185347+0000", 2024-04-18T22:24:52.200 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7614999999999997e-05 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185463+0000", 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000115931 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185571+0000", 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010780700000000001 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253183 5.c 5:34e58fe4:::benchmark_data_smithi032_22800_object125069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.185784+0000", 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.001849051, 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0021653610000000002, 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253183 2024-04-18T22:24:52.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185784+0000", 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185784+0000", 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185785+0000", 2024-04-18T22:24:52.203 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8599999999999997e-07 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185789+0000", 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1540000000000004e-06 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185801+0000", 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.163e-05 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.185808+0000", 2024-04-18T22:24:52.204 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0099999999999998e-06 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186247+0000", 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00043856 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186313+0000", 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6088999999999997e-05 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186370+0000", 2024-04-18T22:24:52.205 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7092999999999997e-05 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253184 5.3 5:c2bbe8f8:::benchmark_data_smithi032_22800_object125070:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.186362+0000", 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0012718409999999999, 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0016555000000000001, 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.206 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253184 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186362+0000", 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186362+0000", 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186362+0000", 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.8400000000000003e-07 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186366+0000", 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.7230000000000002e-06 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186382+0000", 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5962999999999999e-05 2024-04-18T22:24:52.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.186388+0000", 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.348e-06 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187123+0000", 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00073468199999999996 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187203+0000", 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9802000000000006e-05 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187293+0000", 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9639e-05 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253186 5.8 5:103d3bb9:::benchmark_data_smithi032_22800_object125072:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.187195+0000", 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00043887, 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00090351799999999996, 2024-04-18T22:24:52.210 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253186 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187195+0000", 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.211 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187195+0000", 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187195+0000", 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6700000000000003e-07 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187199+0000", 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.1500000000000001e-06 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.212 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187207+0000", 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6120000000000001e-06 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187213+0000", 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4239999999999998e-06 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187260+0000", 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6223000000000001e-05 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.213 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187386+0000", 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000126718 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop(client.4586.0:253185 5.5 e68/49 5:a2fd4d5c:::benchmark_data_smithi032_22800_object125071:head v 68'15665, mlcod=68'15663)", 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.187245+0000", 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00038876399999999999, 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00091398399999999996, 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187245+0000", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187245+0000", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187245+0000", 2024-04-18T22:24:52.215 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2100000000000002e-07 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187250+0000", 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8659999999999997e-06 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187257+0000", 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.8229999999999997e-06 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.216 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187262+0000", 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7559999999999999e-06 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187351+0000", 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9456000000000005e-05 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187371+0000", 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9656999999999998e-05 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.217 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253188 5.7 5:e049fadb:::benchmark_data_smithi032_22800_object125074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.187383+0000", 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00025076999999999999, 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00083258900000000001, 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253188 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187383+0000", 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187383+0000", 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187383+0000", 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9500000000000002e-07 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187386+0000", 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.819e-06 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187392+0000", 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7740000000000001e-06 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.220 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187396+0000", 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9149999999999998e-06 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187440+0000", 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3791999999999997e-05 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187539+0000", 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.9510000000000001e-05 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187621+0000", 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2175999999999994e-05 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253189 5.e 5:7fb710c1:::benchmark_data_smithi032_22800_object125075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.187445+0000", 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.00018843700000000001, 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00083608100000000002, 2024-04-18T22:24:52.222 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253189 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187445+0000", 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187445+0000", 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187446+0000", 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187449+0000", 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9849999999999998e-06 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187456+0000", 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6120000000000001e-06 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187462+0000", 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.8860000000000001e-06 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187511+0000", 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8433000000000002e-05 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187633+0000", 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012241699999999999 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:253180 5.6 e68/19 ondisk, result = 0)", 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.187868+0000", 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997649, 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00047177100000000001, 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "started", 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187868+0000", 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187868+0000", 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187869+0000", 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.2099999999999998e-07 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.227 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187871+0000", 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0980000000000001e-06 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187877+0000", 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3589999999999996e-06 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187882+0000", 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8659999999999997e-06 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.228 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188304+0000", 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00042245899999999997 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188313+0000", 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.5860000000000005e-06 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.229 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:253190 5.2 5.9cc1ff62 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.187897+0000", 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9997368, 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00049689899999999997, 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 253190 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.230 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187897+0000", 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187897+0000", 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187897+0000", 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.6799999999999999e-07 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.231 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187902+0000", 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8919999999999999e-06 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187912+0000", 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.5200000000000003e-06 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.187918+0000", 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6030000000000001e-06 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.232 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:253182 5.3 e68/19)", 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.188025+0000", 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.999608, 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000415375, 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188025+0000", 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188025+0000", 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188026+0000", 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2e-07 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188028+0000", 2024-04-18T22:24:52.234 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5280000000000002e-06 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188035+0000", 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1620000000000002e-06 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188041+0000", 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1420000000000001e-06 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_repop_reply(client.4586.0:253183 5.c e68/19)", 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:52.188252+0000", 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 4294967295.9993815, 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00022930499999999999, 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "queued for pg", 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188252+0000", 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.236 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188252+0000", 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188252+0000", 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3099999999999998e-07 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188255+0000", 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0580000000000002e-06 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.237 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188263+0000", 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9449999999999993e-06 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:52.188269+0000", 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.7440000000000004e-06 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 13 2024-04-18T22:24:52.238 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:52.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:52.240 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:52.239+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:52.340 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:52.339+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-04-18T22:24:52.425 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:52.425 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-04-18T22:24:52.425 INFO:teuthology.orchestra.run.smithi086.stdout: "complaint_time": 30, 2024-04-18T22:24:52.425 INFO:teuthology.orchestra.run.smithi086.stdout: "num_blocked_ops": 0 2024-04-18T22:24:52.425 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:52.442 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:52.439+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:52.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:52.542 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:52.541+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:52.642 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-18T22:24:52.642 INFO:teuthology.orchestra.run.smithi086.stdout: "size": 20, 2024-04-18T22:24:52.642 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 600, 2024-04-18T22:24:52.642 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [ 2024-04-18T22:24:52.642 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137323 5.0 5:0a62c76f:::benchmark_data_smithi032_22800_object9209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.604890675, 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.11295535700000001, 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137323 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.643 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035574+0000", 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035576+0000", 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.0030000000000001e-06 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035587+0000", 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.023e-05 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035612+0000", 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5128999999999999e-05 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035628+0000", 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.5866e-05 2024-04-18T22:24:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035764+0000", 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013683799999999999 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.035789+0000", 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4275999999999999e-05 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137808+0000", 2024-04-18T22:24:52.646 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102018918 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.137976+0000", 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016888100000000001 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.138251+0000", 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00027483199999999997 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144609+0000", 2024-04-18T22:24:52.647 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0063577659999999999 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148150+0000", 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0035412799999999999 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148265+0000", 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114936 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148530+0000", 2024-04-18T22:24:52.648 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00026440200000000001 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137325 5.0 5:0346b8b1:::benchmark_data_smithi032_22800_object9211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.600247676000002, 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.111136737, 2024-04-18T22:24:52.649 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137325 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040217+0000", 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040220+0000", 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4399999999999999e-06 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040233+0000", 2024-04-18T22:24:52.651 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2921999999999999e-05 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040263+0000", 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.0439000000000001e-05 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040288+0000", 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5219e-05 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.652 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040438+0000", 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014979500000000001 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.040460+0000", 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2048999999999998e-05 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141597+0000", 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10113680899999999 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.653 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.141815+0000", 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000217798 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.142031+0000", 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00021598400000000001 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151181+0000", 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0091506969999999993 2024-04-18T22:24:52.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151199+0000", 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.7319000000000001e-05 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151247+0000", 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8455000000000003e-05 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151354+0000", 2024-04-18T22:24:52.655 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106811 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137334 5.0 5:04fc6637:::benchmark_data_smithi032_22800_object9220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.591149938999997, 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.102242213, 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137334 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.657 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049315+0000", 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049317+0000", 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.6029999999999999e-06 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049325+0000", 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3389999999999993e-06 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049350+0000", 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5369000000000001e-05 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049364+0000", 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3655000000000001e-05 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049462+0000", 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8451999999999999e-05 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.049477+0000", 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4735999999999999e-05 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144713+0000", 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095236223999999994 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.144919+0000", 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000205999 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.145126+0000", 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00020699100000000001 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151371+0000", 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0062440769999999998 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151426+0000", 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5859000000000002e-05 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.661 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151475+0000", 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8222e-05 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151557+0000", 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2687000000000003e-05 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137343 5.0 5:08f84eeb:::benchmark_data_smithi032_22800_object9229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.582130014000001, 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.100709696, 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137343 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058335+0000", 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058337+0000", 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.86e-06 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058345+0000", 2024-04-18T22:24:52.664 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9200000000000004e-06 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058367+0000", 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2620999999999999e-05 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058389+0000", 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.1444999999999999e-05 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058533+0000", 2024-04-18T22:24:52.665 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000144499 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.058558+0000", 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5058999999999998e-05 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148612+0000", 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090053174 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.666 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.148833+0000", 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000220998 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.149023+0000", 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00019072199999999999 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158915+0000", 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0098914180000000008 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.667 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158959+0000", 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4002999999999999e-05 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.158993+0000", 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4687000000000002e-05 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.159045+0000", 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1289999999999999e-05 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137355 5.0 5:0427d8b1:::benchmark_data_smithi032_22800_object9241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.566118646, 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101482883, 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137355 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.669 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074346+0000", 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074348+0000", 2024-04-18T22:24:52.670 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9369999999999998e-06 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074359+0000", 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0383e-05 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074386+0000", 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.7046000000000002e-05 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074400+0000", 2024-04-18T22:24:52.671 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.4756000000000001e-05 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074524+0000", 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000123886 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.074538+0000", 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.313e-05 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151639+0000", 2024-04-18T22:24:52.672 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.077101876999999999 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151777+0000", 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013736799999999999 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.151884+0000", 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107339 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172670+0000", 2024-04-18T22:24:52.673 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.020785597999999999 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175679+0000", 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.003009477 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175735+0000", 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5535999999999998e-05 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.674 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175829+0000", 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.4549999999999994e-05 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:137359 5.0 5:07ffc1ca:::benchmark_data_smithi032_22800_object9245:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 49.563315226, 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.099033607999999995, 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.675 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 137359 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077150+0000", 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077151+0000", 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.3400000000000001e-06 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077157+0000", 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3369999999999996e-06 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077176+0000", 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.8525e-05 2024-04-18T22:24:52.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077187+0000", 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.0804e-05 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077326+0000", 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013942100000000001 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for peered", 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.077352+0000", 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5833000000000002e-05 2024-04-18T22:24:52.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153832+0000", 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.076479758999999994 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.153998+0000", 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000166403 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.154121+0000", 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012259900000000001 2024-04-18T22:24:52.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.172683+0000", 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.018562171999999998 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.175923+0000", 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00323982 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176019+0000", 2024-04-18T22:24:52.680 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6532000000000003e-05 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:03.176183+0000", 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00016406299999999999 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203298 5.2 5:477e3e4c:::benchmark_data_smithi032_22800_object75184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.753816262000001, 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10500430400000001, 2024-04-18T22:24:52.681 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203298 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886649+0000", 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.9800000000000003e-07 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886653+0000", 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.241e-06 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886661+0000", 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2649999999999996e-06 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.886665+0000", 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4800000000000003e-06 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890382+0000", 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0037164189999999999 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890524+0000", 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014180599999999999 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890644+0000", 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120141 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894960+0000", 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043163510000000004 2024-04-18T22:24:52.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991564+0000", 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.096604314999999996 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991602+0000", 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8089000000000001e-05 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991653+0000", 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0599000000000001e-05 2024-04-18T22:24:52.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203299 5.a 5:5b381091:::benchmark_data_smithi032_22800_object75185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.752150056000001, 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.105728395, 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203299 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888315+0000", 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.688 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888316+0000", 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.8100000000000001e-07 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888320+0000", 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3279999999999999e-06 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888330+0000", 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.6379999999999999e-06 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.689 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.888336+0000", 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5719999999999999e-06 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892568+0000", 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0042318440000000002 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.892699+0000", 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00013117099999999999 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.690 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993928+0000", 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101228735 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.993972+0000", 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.4159999999999997e-05 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994043+0000", 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.0965999999999998e-05 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203302 5.a 5:51847fa5:::benchmark_data_smithi032_22800_object75188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.751177559999999, 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.104895119, 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203302 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889287+0000", 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889288+0000", 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.8299999999999997e-07 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.693 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889290+0000", 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3939999999999999e-06 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889297+0000", 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.1559999999999998e-06 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.889302+0000", 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7639999999999996e-06 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895026+0000", 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0057237269999999996 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895151+0000", 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000124692 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994052+0000", 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098901279999999994 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994095+0000", 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.3115000000000003e-05 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994183+0000", 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7508000000000003e-05 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203305 5.2 5:4d97f250:::benchmark_data_smithi032_22800_object75191:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.696 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.749817582999999, 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.103584204, 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203305 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:52.697 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890647+0000", 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890648+0000", 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.09e-07 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890653+0000", 2024-04-18T22:24:52.698 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.1440000000000002e-06 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890679+0000", 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.5778e-05 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.890688+0000", 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.3799999999999994e-06 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895068+0000", 2024-04-18T22:24:52.699 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0043804849999999999 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895137+0000", 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.9406999999999995e-05 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895200+0000", 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2677999999999999e-05 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989117+0000", 2024-04-18T22:24:52.700 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093917374999999997 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994188+0000", 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.005070626 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994213+0000", 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.4417e-05 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994232+0000", 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.9105000000000001e-05 2024-04-18T22:24:52.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203310 5.2 5:42d695e2:::benchmark_data_smithi032_22800_object75196:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.747250452999999, 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.101077819, 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.702 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203310 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893215+0000", 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.5799999999999999e-07 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893219+0000", 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6720000000000002e-06 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893228+0000", 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9330000000000007e-06 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.893233+0000", 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.0749999999999997e-06 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898519+0000", 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0052856439999999999 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898586+0000", 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7774000000000005e-05 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.705 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.898648+0000", 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2145999999999998e-05 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989153+0000", 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090504876999999997 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994235+0000", 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0050813300000000002 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.706 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994258+0000", 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.3765999999999999e-05 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994292+0000", 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3943999999999999e-05 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203311 5.2 5:49c221f5:::benchmark_data_smithi032_22800_object75197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.746195814, 2024-04-18T22:24:52.707 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10212882700000001, 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203311 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894269+0000", 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894270+0000", 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.2799999999999996e-07 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894273+0000", 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.3000000000000002e-06 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894281+0000", 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.5349999999999999e-06 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.894285+0000", 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.6569999999999996e-06 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989181+0000", 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094895811999999996 2024-04-18T22:24:52.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989507+0000", 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00032610700000000001 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989647+0000", 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00014005300000000001 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994300+0000", 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0046525239999999999 2024-04-18T22:24:52.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996308+0000", 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0020080480000000001 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996360+0000", 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2039000000000001e-05 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.996398+0000", 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8124000000000003e-05 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203313 5.6 5:65852246:::benchmark_data_smithi032_22800_object75199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.745353873999999, 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095219751000000005, 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203313 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895111+0000", 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.714 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895112+0000", 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4300000000000003e-07 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895115+0000", 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6179999999999999e-06 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895122+0000", 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.6189999999999996e-06 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.715 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895127+0000", 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.5449999999999997e-06 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895182+0000", 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5120000000000001e-05 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895299+0000", 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00011754199999999999 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.716 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.895407+0000", 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010746 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.899393+0000", 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039858949999999997 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989659+0000", 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.090265972 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.717 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990276+0000", 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00061694799999999998 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.990331+0000", 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.5389000000000001e-05 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203315 5.8 5:1ad93081:::benchmark_data_smithi032_22800_object75201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:52.718 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.743540763999999, 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097958515999999995, 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203315 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896924+0000", 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896925+0000", 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.9599999999999998e-07 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896930+0000", 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.7180000000000004e-06 2024-04-18T22:24:52.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896940+0000", 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 9.662e-06 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896946+0000", 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4500000000000001e-06 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.896984+0000", 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.8062999999999998e-05 2024-04-18T22:24:52.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897059+0000", 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4794000000000003e-05 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994712+0000", 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097653161000000002 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994776+0000", 2024-04-18T22:24:52.722 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4178999999999997e-05 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994883+0000", 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000106593 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203316 5.2 5:4c2e1dfd:::benchmark_data_smithi032_22800_object75202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.743353713000001, 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.10004213200000001, 2024-04-18T22:24:52.723 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203316 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.724 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897111+0000", 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897112+0000", 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6400000000000001e-07 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897118+0000", 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3280000000000002e-06 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.725 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897127+0000", 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.7020000000000007e-06 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897133+0000", 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.3110000000000002e-06 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991716+0000", 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094582845999999998 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.726 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991841+0000", 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000125036 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "waiting for subops from 2", 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991956+0000", 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000114679 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994302+0000", 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0023464079999999999 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.727 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "sub_op_commit_rec", 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997066+0000", 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0027637500000000001 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997105+0000", 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.9147999999999999e-05 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.997153+0000", 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.816e-05 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203319 5.4 5:2cd790a9:::benchmark_data_smithi032_22800_object75205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.743114991999999, 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095588554000000006, 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203319 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.729 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897350+0000", 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897351+0000", 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7400000000000002e-07 2024-04-18T22:24:52.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897355+0000", 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.0069999999999999e-06 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897363+0000", 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4749999999999993e-06 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897369+0000", 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.0190000000000003e-06 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897443+0000", 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3219999999999994e-05 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897563+0000", 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000120297 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992782+0000", 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095218784000000001 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.732 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992864+0000", 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.2757999999999996e-05 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.992939+0000", 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.4220000000000004e-05 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203320 5.a 5:59b85fae:::benchmark_data_smithi032_22800_object75206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:52.733 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.742925787000001, 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.098417163000000002, 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203320 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:52.734 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897539+0000", 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897541+0000", 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.2170000000000001e-06 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897545+0000", 2024-04-18T22:24:52.735 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.4809999999999998e-06 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897554+0000", 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.8489999999999995e-06 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897562+0000", 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.0220000000000004e-06 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991694+0000", 2024-04-18T22:24:52.736 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094132258999999996 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991824+0000", 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000129498 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995795+0000", 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0039718870000000003 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995852+0000", 2024-04-18T22:24:52.737 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.6589999999999999e-05 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995956+0000", 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00010436 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203321 5.e 5:7b6fb23a:::benchmark_data_smithi032_22800_object75207:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.742814249999999, 2024-04-18T22:24:52.738 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.094119515000000001, 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203321 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.739 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897651+0000", 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7999999999999995e-07 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897654+0000", 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 2.9979999999999999e-06 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897662+0000", 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.7510000000000008e-06 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897670+0000", 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.9179999999999994e-06 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897735+0000", 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.4833999999999996e-05 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897907+0000", 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00017168700000000001 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991582+0000", 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.093674898000000006 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991666+0000", 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 8.4530999999999994e-05 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.991770+0000", 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000104218 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203322 5.4 5:22ec6e57:::benchmark_data_smithi032_22800_object75208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.742637425000002, 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.097755745000000005, 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.743 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203322 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.744 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897828+0000", 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3300000000000001e-07 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897832+0000", 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 3.6150000000000001e-06 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897840+0000", 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.6639999999999998e-06 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.745 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.897847+0000", 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3189999999999997e-06 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989116+0000", 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.091269496000000006 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989431+0000", 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00031452599999999998 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995473+0000", 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0060421110000000002 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995516+0000", 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 4.2567000000000001e-05 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995583+0000", 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 6.7713999999999996e-05 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.747 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "description": "osd_op(client.4586.0:203324 5.8 5:14c8c3c0:::benchmark_data_smithi032_22800_object75210:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "initiated_at": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 21.740461700000001, 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.095050082999999994, 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "continuous": false, 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type_data": { 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "client_info": { 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "client": "client.4586", 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tid": 203324 2024-04-18T22:24:52.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "events": [ 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "initiated", 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "header_read", 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900003+0000", 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "throttled", 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900004+0000", 2024-04-18T22:24:52.749 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 7.3e-07 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "all_read", 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.900009+0000", 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.2360000000000004e-06 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "dispatched", 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988785+0000", 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.088775346000000005 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.750 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.988797+0000", 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 1.2612000000000001e-05 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "reached_pg", 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989353+0000", 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00055552800000000001 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "started", 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.989480+0000", 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.00012677500000000001 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "op_commit", 2024-04-18T22:24:52.751 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994891+0000", 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.0054119279999999999 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "commit_sent", 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.994946+0000", 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 5.486e-05 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "event": "done", 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "time": "2024-04-18T22:24:30.995053+0000", 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: "duration": 0.000107068 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.752 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-18T22:24:52.753 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-18T22:24:52.753 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-18T22:24:52.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:52.755 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:52.749+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:52.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:52.853+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: "ops": [ 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:254956 5.c e69/19 5:3ad36471:::benchmark_data_smithi032_22800_object126842:head v 69'15934, mlcod=69'15931)", 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.901892+0000", 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0028915070000000002, 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0029162760000000002, 2024-04-18T22:24:52.904 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901892+0000", 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901892+0000", 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.905 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901893+0000", 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.1200000000000002e-07 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901897+0000", 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.69e-06 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901909+0000", 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1652e-05 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.906 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901916+0000", 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.6170000000000003e-06 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901966+0000", 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.0494000000000001e-05 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901985+0000", 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8566000000000001e-05 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.907 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:254955 5.3 e69/19 5:c3e3562e:::benchmark_data_smithi032_22800_object126841:head v 69'16275, mlcod=69'16270)", 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.902968+0000", 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0018155160000000001, 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0018967470000000001, 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.902968+0000", 2024-04-18T22:24:52.908 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.902968+0000", 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.902969+0000", 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.2500000000000005e-07 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.902976+0000", 2024-04-18T22:24:52.909 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.0960000000000003e-06 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.902988+0000", 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1909999999999999e-05 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.902995+0000", 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6650000000000003e-06 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903041+0000", 2024-04-18T22:24:52.910 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.5614000000000001e-05 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903059+0000", 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8332999999999999e-05 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:254957 5.7 e69/23 5:ed9ef785:::benchmark_data_smithi032_22800_object126843:head v 69'16049, mlcod=69'16046)", 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.903245+0000", 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.001538457, 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.001651404, 2024-04-18T22:24:52.911 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903245+0000", 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903245+0000", 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.912 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903246+0000", 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4300000000000003e-07 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903252+0000", 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.9639999999999997e-06 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903261+0000", 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4970000000000001e-06 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.913 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903265+0000", 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.4460000000000003e-06 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903318+0000", 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2907e-05 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.903339+0000", 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.0866999999999999e-05 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.914 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:254960 5.5 5:aa24e956:::benchmark_data_smithi032_22800_object126846:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.904257+0000", 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00052671499999999997, 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00067225200000000003, 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 254960 2024-04-18T22:24:52.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904257+0000", 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904257+0000", 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904257+0000", 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.1699999999999998e-07 2024-04-18T22:24:52.916 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904260+0000", 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.745e-06 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904269+0000", 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.7630000000000006e-06 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904274+0000", 2024-04-18T22:24:52.917 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.668e-06 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904320+0000", 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6400999999999998e-05 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904466+0000", 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0001461 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.918 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904589+0000", 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00012307 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:254962 5.1 5:81c14e75:::benchmark_data_smithi032_22800_object126848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.904421+0000", 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.000362433, 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00054900299999999995, 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "waiting for sub ops", 2024-04-18T22:24:52.919 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 254962 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904421+0000", 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904421+0000", 2024-04-18T22:24:52.920 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904422+0000", 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.6300000000000004e-07 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904427+0000", 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.9459999999999997e-06 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904435+0000", 2024-04-18T22:24:52.921 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.1969999999999996e-06 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904440+0000", 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4e-06 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904492+0000", 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2034999999999999e-05 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904615+0000", 2024-04-18T22:24:52.922 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000122757 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "waiting for subops from 0", 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904734+0000", 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.000118591 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:254958 5.3 e69/19 5:ca64b1aa:::benchmark_data_smithi032_22800_object126844:head v 69'16276, mlcod=69'16272)", 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.904611+0000", 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00017246000000000001, 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00039576100000000001, 2024-04-18T22:24:52.923 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904611+0000", 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904611+0000", 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.924 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904612+0000", 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4499999999999997e-07 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904617+0000", 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.1100000000000002e-06 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904625+0000", 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 8.4780000000000008e-06 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.925 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904632+0000", 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.075e-06 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904685+0000", 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.2531000000000002e-05 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.904703+0000", 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.8487000000000001e-05 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.926 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_op(client.4586.0:254953 5.d 5:bea4017f:::benchmark_data_smithi032_22800_object126839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.900997+0000", 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.00378613, 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0040417300000000003, 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "client_info": { 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "client": "client.4586", 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "client_addr": "172.21.15.32:0/1416672495", 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 254953 2024-04-18T22:24:52.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.900997+0000", 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.900997+0000", 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.900998+0000", 2024-04-18T22:24:52.928 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.3099999999999996e-07 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901002+0000", 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.8589999999999998e-06 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901016+0000", 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.3725e-05 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901025+0000", 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.1980000000000007e-06 2024-04-18T22:24:52.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901433+0000", 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00040755700000000001 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901533+0000", 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.00010042700000000001 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.930 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:254951 5.3 e69/19 5:c2bfb711:::benchmark_data_smithi032_22800_object126837:head v 69'16274, mlcod=69'16270)", 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.901422+0000", 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033611660000000001, 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036526990000000001, 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901422+0000", 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.931 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901422+0000", 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901423+0000", 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 5.4700000000000001e-07 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901428+0000", 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.6990000000000002e-06 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.932 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901437+0000", 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 9.5799999999999998e-06 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901454+0000", 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.6943999999999999e-05 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901503+0000", 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.8724999999999999e-05 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.933 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901514+0000", 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 1.1430999999999999e-05 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "description": "osd_repop(client.4586.0:254952 5.7 e69/23 5:e99494af:::benchmark_data_smithi032_22800_object126838:head v 69'16048, mlcod=69'16046)", 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "initiated_at": "2024-04-18T22:24:52.901471+0000", 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "age": 0.0033125559999999999, 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0.0036344659999999998, 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "continuous": false, 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "type_data": { 2024-04-18T22:24:52.934 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "started", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901471+0000", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901471+0000", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 0 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901471+0000", 2024-04-18T22:24:52.935 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.6699999999999999e-07 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read", 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901475+0000", 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 3.7510000000000002e-06 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched", 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901482+0000", 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 7.3799999999999996e-06 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "queued_for_pg", 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901489+0000", 2024-04-18T22:24:52.936 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 6.5849999999999996e-06 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "reached_pg", 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901533+0000", 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 4.3578000000000002e-05 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "started", 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-18T22:24:52.901554+0000", 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "duration": 2.1321000000000001e-05 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-18T22:24:52.937 INFO:teuthology.orchestra.run.smithi196.stdout: "num_ops": 9 2024-04-18T22:24:52.938 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-18T22:24:52.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:52.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T22:24:52.955 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:52.955+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:53.056 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:53.053+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:53.110 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-04-18T22:24:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T22:24:53.111 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-04-18T22:24:53.130 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-04-18T22:24:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T22:24:53.132 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T22:24:53.133 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T22:24:53.136 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:53.136 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:53.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:53.157 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:53.155+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:53.258 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:53.255+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:53.350 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:53.351 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:53.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:53.357+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:53.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:53.425 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Removed 128111 objects 2024-04-18T22:24:53.425 INFO:tasks.radosbench.radosbench.0.smithi032.stdout:Clean up completed and total clean up time :56.2209 2024-04-18T22:24:53.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:53.457+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:53.488 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:53.488 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:53.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:53.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:53.561+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:53.609 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:53.610 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:53.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:53.667 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:53.667+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:53.734 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:53.735 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:53.747 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:53.791 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:53.791+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:53.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:53.861+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:53.879 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:53.879 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:53.890 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:53.975 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:53.973+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:54.053 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:54.053 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:54.065 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:54.063+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:54.066 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:54.143 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-18T22:24:54.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-18T22:24:54.167 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:54.167+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:54.230 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:54.231 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:54.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T22:24:54.268 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-18T22:24:54.265+0000 7f57eb037640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 21723) UID: 0 2024-04-18T22:24:54.369 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:54.367+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:54.371 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:54.371 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:54.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T22:24:54.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T22:24:54.471 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-18T22:24:54.471+0000 7fa2991f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 21649) UID: 0 2024-04-18T22:24:54.514 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'unique_pool_0' removed 2024-04-18T22:24:54.538 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-18T22:24:54.554 INFO:tasks.thrashosds:joining thrashosds 2024-04-18T22:24:54.566 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:54.566 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:54.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T22:24:54.771 INFO:teuthology.orchestra.run.smithi086.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:54.771 INFO:teuthology.orchestra.run.smithi086.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:54.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T22:24:54.922 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:54.922 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:54.938 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T22:24:55.055 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:55.055 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:55.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T22:24:55.214 INFO:teuthology.orchestra.run.smithi196.stdout:ERROR: (22) Invalid argument 2024-04-18T22:24:55.214 INFO:teuthology.orchestra.run.smithi196.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T22:24:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-04-18T22:24:56.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~82s 2024-04-18T22:24:57.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T22:24:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-04-18T22:24:58.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T22:24:59.566 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-04-18T22:25:02.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~88s 2024-04-18T22:25:07.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~94s 2024-04-18T22:25:13.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~99s 2024-04-18T22:25:19.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~105s 2024-04-18T22:25:25.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~111s 2024-04-18T22:25:31.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~117s 2024-04-18T22:25:36.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~123s 2024-04-18T22:25:42.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~129s 2024-04-18T22:25:48.580 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~134s 2024-04-18T22:25:54.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2024-04-18T22:26:00.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~146s 2024-04-18T22:26:06.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~152s 2024-04-18T22:26:11.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~158s 2024-04-18T22:26:14.601 DEBUG:teuthology.orchestra.console.smithi096:expect before: [Enter `^Ec?' for help] \x80 \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent GE v1.5.13 PXE 2.1 Build 092 (WfM 2.0) Press Ctrl+S to enter the Setup Menu.                       Press Ctrl+S to enter the Setup Menu..  \x80 \x80 \x80 \x80  Intel(R) Boot Agent XE v2.1.60 Copyright (C) 1997-2009, Intel Corporation  CLIENT MAC ADDR: 0C C4 7A 8F CC 0D GUID: 00000000 0000 0000 0000 0CC47AD8D9BC DHCP./                    DHCP.| DHCP./ DHCP.- DHCP.\ DHCP.| DHCP./ CLIENT IP: 172.21.15.96 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 GATEWAY IP: 172.21.15.254 PXE->EB: !PXE at 96D3:0070, entry point at 96D3:0106  UNDI code segment 96D3:3E70, data segment 8F94:73F0 (574-619kB)  UNDI device is PCI 03:00.1, type DIX+802.3  574kB free base memory after PXE unload iPXE initialising devices... iPXE initialising devices...ok iPXE 1.20.1+ (g4bd0) -- Open Source Network Boot Firmware -- http://ipxe.org Features: DNS FTP HTTP HTTPS iSCSI NFS TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT Configuring (net0 0c:c4:7a:8f:cc:0d)... Configuring (net0 0c:c4:7a:8f:cc:0d).... Configuring (net0 0c:c4:7a:8f:cc:0d)..... Configuring (net0 0c:c4:7a:8f:cc:0d)...... ok Received DHCP answer on interface net0  GNU GRUB version 2.04   ******************************************************************************  * Ubuntu *  **Advanced options for Ubuntu *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  ******************************************************************************   Use the * and * keys to select which entry is highlighted.  Press enter to boot the selected OS, `e' to edit the commands  before booting or `c' for a command-line.  The highlighted entry will be executed automatically in 4s.  The highlighted entry will be executed automatically in 3s.  The highlighted entry will be executed automatically in 2s.  The highlighted entry will be executed automatically in 1s.                     Loading Linux 5.15.0-102-generic ... Loading initial ramdisk ... [ 0.000000] microcode: microcode updated early to revision 0xb000040, date = 2021-05-19 [ 0.000000] Linux version 5.15.0-102-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 (Ubuntu 5.15.0-102.112-generic 5.15.148) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic root=UUID=7e9e2103-8bb7-4452-b35a-3b1230743254 ro pci=realloc=off console=tty0 console=ttyS1,115200 [ 0.000000] KERNEL supported cpus: [ 0.000000] Intel GenuineIntel [ 0.000000] AMD AuthenticAMD [ 0.000000] Hygon HygonGenuine [ 0.000000] Centaur CentaurHauls [ 0.000000] zhaoxin Shanghai [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bbff] usable [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078ec1fff] usable [ 0.000000] BIOS-e820: [mem 0x0000000078f74000-0x00000000798adfff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000798ae000-0x0000000079d4afff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000079d4b000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0 present. [ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0 12/17/2015 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 3499.750 MHz processor [ 0.000412] last_pfn = 0x880000 max_arch_pfn = 0x400000000 [ 0.000592] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.001880] last_pfn = 0x78ec2 max_arch_pfn = 0x400000000 [ 0.008227] found SMP MP-table at [mem 0x000fda20-0x000fda2f] [ 0.008240] Using GB pages for direct mapping [ 0.008324] RAMDISK: [mem 0x300cf000-0x3405efff] [ 0.008330] ACPI: Early table checksum verification disabled [ 0.008333] ACPI: RSDP 0x00000000000F0580 000024 (v02 SUPERM) [ 0.008337] ACPI: XSDT 0x00000000798AE098 0000B4 (v01 01072009 AMI 00010013) [ 0.008342] ACPI: FACP 0x00000000798E15B0 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008347] ACPI: DSDT 0x00000000798AE1E8 0333C4 (v02 SUPERM SMCI--MB 01072009 INTL 20091013) [ 0.008350] ACPI: FACS 0x0000000079D49F80 000040 [ 0.008353] ACPI: APIC 0x00000000798E16C0 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008355] ACPI: FPDT 0x00000000798E1788 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008358] ACPI: FIDT 0x00000000798E17D0 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008361] ACPI: SPMI 0x00000000798E1870 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000) [ 0.008364] ACPI: MCFG 0x00000000798E18B0 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097) [ 0.008367] ACPI: UEFI 0x00000000798E18F0 000042 (v01 SUPERM SMCI--MB 01072009 00000000) [ 0.008369] ACPI: HPET 0x00000000798E1938 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.008372] ACPI: WDDT 0x00000000798E1970 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013) [ 0.008375] ACPI: SSDT 0x00000000798E19B0 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913) [ 0.008378] ACPI: SSDT 0x00000000798F8B30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913) [ 0.008381] ACPI: SSDT 0x00000000798FB180 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913) [ 0.008384] ACPI: PRAD 0x00000000798FB1E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913) [ 0.008386] ACPI: DMAR 0x00000000798FB2F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.008389] ACPI: HEST 0x00000000798FB3B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008392] ACPI: BERT 0x00000000798FB630 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008395] ACPI: ERST 0x00000000798FB660 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008398] ACPI: EINJ 0x00000000798FB890 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008400] ACPI: Reserving FACP table memory at [mem 0x798e15b0-0x798e16bb] [ 0.008402] ACPI: Reserving DSDT table memory at [mem 0x798ae1e8-0x798e15ab] [ 0.008403] ACPI: Reserving FACS table memory at [mem 0x79d49f80-0x79d49fbf] [ 0.008404] ACPI: Reserving APIC table memory at [mem 0x798e16c0-0x798e1787] [ 0.008405] ACPI: Reserving FPDT table memory at [mem 0x798e1788-0x798e17cb] [ 0.008406] ACPI: Reserving FIDT table memory at [mem 0x798e17d0-0x798e186b] [ 0.008407] ACPI: Reserving SPMI table memory at [mem 0x798e1870-0x798e18af] [ 0.008408] ACPI: Reserving MCFG table memory at [mem 0x798e18b0-0x798e18eb] [ 0.008409] ACPI: Reserving UEFI table memory at [mem 0x798e18f0-0x798e1931] [ 0.008410] ACPI: Reserving HPET table memory at [mem 0x798e1938-0x798e196f] [ 0.008411] ACPI: Reserving WDDT table memory at [mem 0x798e1970-0x798e19af] [ 0.008412] ACPI: Reserving SSDT table memory at [mem 0x798e19b0-0x798f8b2e] [ 0.008413] ACPI: Reserving SSDT table memory at [mem 0x798f8b30-0x798fb17b] [ 0.008414] ACPI: Reserving SSDT table memory at [mem 0x798fb180-0x798fb1e3] [ 0.008415] ACPI: Reserving PRAD table memory at [mem 0x798fb1e8-0x798fb2e9] [ 0.008416] ACPI: Reserving DMAR table memory at [mem 0x798fb2f0-0x798fb3ab] [ 0.008417] ACPI: Reserving HEST table memory at [mem 0x798fb3b0-0x798fb62b] [ 0.008418] ACPI: Reserving BERT table memory at [mem 0x798fb630-0x798fb65f] [ 0.008419] ACPI: Reserving ERST table memory at [mem 0x798fb660-0x798fb88f] [ 0.008420] ACPI: Reserving EINJ table memory at [mem 0x798fb890-0x798fb9bf] [ 0.008514] No NUMA configuration found [ 0.008515] Faking a node at [mem 0x0000000000000000-0x000000087fffffff] [ 0.008524] NODE_DATA(0) allocated [mem 0x87ffd6000-0x87fffffff] [ 0.008855] Zone ranges: [ 0.008856] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.008858] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.008860] Normal [mem 0x0000000100000000-0x000000087fffffff] [ 0.008861] Device empty [ 0.008863] Movable zone start for each node [ 0.008865] Early memory node ranges [ 0.008866] node 0: [mem 0x0000000000001000-0x000000000008afff] [ 0.008868] node 0: [mem 0x0000000000100000-0x0000000078ec1fff] [ 0.008869] node 0: [mem 0x0000000100000000-0x000000087fffffff] [ 0.008873] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff] [ 0.008878] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.008913] On node 0, zone DMA: 117 pages in unavailable ranges [ 0.077944] On node 0, zone Normal: 28990 pages in unavailable ranges [ 0.078059] ACPI: PM-Timer IO Port: 0x408 [ 0.078066] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) [ 0.078068] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) [ 0.078069] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) [ 0.078070] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1]) [ 0.078071] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) [ 0.078071] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) [ 0.078073] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1]) [ 0.078073] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1]) [ 0.078083] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 [ 0.078087] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 [ 0.078089] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.078091] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.078095] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.078096] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 0.078100] TSC deadline timer available [ 0.078101] smpboot: Allowing 8 CPUs, 0 hotplug CPUs [ 0.078111] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.078113] PM: hibernation: Registered nosave memory: [mem 0x0008b000-0x000dffff] [ 0.078114] PM: hibernation: Registered nosave memory: [mem 0x000e0000-0x000fffff] [ 0.078115] PM: hibernation: Registered nosave memory: [mem 0x78ec2000-0x78f73fff] [ 0.078116] PM: hibernation: Registered nosave memory: [mem 0x78f74000-0x798adfff] [ 0.078117] PM: hibernation: Registered nosave memory: [mem 0x798ae000-0x79d4afff] [ 0.078118] PM: hibernation: Registered nosave memory: [mem 0x79d4b000-0x8fffffff] [ 0.078119] PM: hibernation: Registered nosave memory: [mem 0x90000000-0xfed1bfff] [ 0.078120] PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0xfed44fff] [ 0.078121] PM: hibernation: Registered nosave memory: [mem 0xfed45000-0xfeffffff] [ 0.078121] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xffffffff] [ 0.078123] [mem 0x90000000-0xfed1bfff] available for PCI devices [ 0.078125] Booting paravirtualized kernel on bare hardware [ 0.078127] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns [ 0.078133] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 [ 0.078344] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u262144 [ 0.078375] Built 1 zonelists, mobility grouping on. Total pages: 8228742 [ 0.078377] Policy zone: Normal [ 0.078378] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic root=UUID=7e9e2103-8bb7-4452-b35a-3b1230743254 ro pci=realloc=off console=tty0 console=ttyS1,115200 [ 0.078437] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic", will be passed to user space. [ 0.080585] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear) [ 0.081728] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear) [ 0.081815] mem auto-init: stack:off, heap alloc:on, heap free:off [ 0.166209] Memory: 32672704K/33438000K available (16393K kernel code, 4395K rwdata, 10948K rodata, 3352K init, 18716K bss, 765036K reserved, 0K cma-reserved) [ 0.166493] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1 [ 0.166508] Kernel/User page tables isolation: enabled [ 0.166539] ftrace: allocating 50694 entries in 199 pages [ 0.186124] ftrace: allocated 199 pages with 5 groups [ 0.186239] rcu: Hierarchical RCU implementation. [ 0.186241] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8. [ 0.186242] Rude variant of Tasks RCU enabled. [ 0.186243] Tracing variant of Tasks RCU enabled. [ 0.186244] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.186245] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 [ 0.189655] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16 [ 0.189853] random: crng init done [ 0.194731] Console: colour VGA+ 80x25 [ 0.205968] printk: console [tty0] enabled [ 1.177251] printk: console [ttyS1] enabled [ 1.181511] ACPI: Core revision 20210730 [ 1.185669] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns [ 1.194894] APIC: Switch to symmetric I/O mode setup [ 1.199939] DMAR: Host address width 46 [ 1.203851] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1 [ 1.209236] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de [ 1.217329] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff [ 1.223667] DMAR: ATSR flags: 0x0 [ 1.228142] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0 [ 1.234476] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0 [ 1.240898] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0 [ 1.247323] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000 [ 1.252790] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [ 1.252790] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [ 1.267971] DMAR-IR: Enabled IRQ remapping in xapic mode [ 1.273351] x2apic: IRQ remapping doesn't support X2APIC mode [ 1.279648] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 1.302901] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x327264bdb86, max_idle_ns: 440795319403 ns [ 1.313515] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.50 BogoMIPS (lpj=13999000) [ 1.317530] CPU0: Thermal monitoring enabled (TM1) [ 1.321566] process: using mwait in idle threads [ 1.325515] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 [ 1.329513] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 [ 1.333515] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 1.337514] Spectre V2 : Mitigation: Retpolines [ 1.341513] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 1.345513] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 1.349513] Spectre V2 : Enabling Restricted Speculation for firmware calls [ 1.353514] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier [ 1.357513] Spectre V2 : User space: Mitigation: STIBP via seccomp and prctl [ 1.361513] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp [ 1.365517] MDS: Mitigation: Clear CPU buffers [ 1.369513] TAA: Mitigation: Clear CPU buffers [ 1.373513] MMIO Stale Data: Mitigation: Clear CPU buffers [ 1.377518] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 1.381513] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 1.385513] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 1.389513] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 1.393513] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 1.415061] Freeing SMP alternatives memory: 44K [ 1.417514] pid_max: default: 32768 minimum: 301 [ 1.421534] LSM: Security Framework initializing [ 1.425521] landlock: Up and running. [ 1.429513] Yama: becoming mindful. [ 1.433530] AppArmor: AppArmor initialized [ 1.437573] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.441553] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.445870] smpboot: Estimated ratio of average max frequency by base frequency (times 1024): 1053 [ 1.449518] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1) [ 1.453621] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver. [ 1.457514] ... version: 3 [ 1.461513] ... bit width: 48 [ 1.465513] ... generic registers: 4 [ 1.469513] ... value mask: 0000ffffffffffff [ 1.473513] ... max period: 00007fffffffffff [ 1.477513] ... fixed-purpose events: 3 [ 1.481513] ... event mask: 000000070000000f [ 1.485573] signal: max sigframe size: 1776 [ 1.489528] rcu: Hierarchical SRCU implementation. [ 1.494230] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 1.497577] smp: Bringing up secondary CPUs ... [ 1.501594] x86: Booting SMP configuration: [ 1.505515] .... node #0, CPUs: #1 #2 #3 #4 [ 1.513594] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details. [ 1.521513] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details. [ 1.525513] MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details. [ 1.529610] #5 #6 #7 [ 1.530064] smp: Brought up 1 node, 8 CPUs [ 1.537515] smpboot: Max logical packages: 1 [ 1.541513] smpboot: Total of 8 processors activated (55996.00 BogoMIPS) [ 1.549633] devtmpfs: initialized [ 1.552925] x86/mm: Memory block size: 128MB [ 1.555112] ACPI: PM: Registering ACPI NVS region [mem 0x798ae000-0x79d4afff] (4837376 bytes) [ 1.557589] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 1.561524] futex hash table entries: 2048 (order: 5, 131072 bytes, linear) [ 1.565559] pinctrl core: initialized pinctrl subsystem [ 1.569603] PM: RTC time: 22:25:38, date: 2024-04-18 [ 1.573619] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 1.577845] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations [ 1.581811] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 1.585844] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 1.589524] audit: initializing netlink subsys (disabled) [ 1.593531] audit: type=2000 audit(1713479137.356:1): state=initialized audit_enabled=0 res=1 [ 1.593628] thermal_sys: Registered thermal governor 'fair_share' [ 1.597516] thermal_sys: Registered thermal governor 'bang_bang' [ 1.601514] thermal_sys: Registered thermal governor 'step_wise' [ 1.605513] thermal_sys: Registered thermal governor 'user_space' [ 1.609513] thermal_sys: Registered thermal governor 'power_allocator' [ 1.613518] EISA bus registered [ 1.620722] cpuidle: using governor ladder [ 1.621515] cpuidle: using governor menu [ 1.625529] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 1.629514] ACPI: bus type PCI registered [ 1.633515] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 1.637568] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000) [ 1.641515] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820 [ 1.645527] PCI: Using configuration type 1 for base access [ 1.650537] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 1.653530] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 1.657514] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 1.661588] ACPI: Added _OSI(Module Device) [ 1.665515] ACPI: Added _OSI(Processor Device) [ 1.669521] ACPI: Added _OSI(3.0 _SCP Extensions) [ 1.673513] ACPI: Added _OSI(Processor Aggregator Device) [ 1.677513] ACPI: Added _OSI(Linux-Dell-Video) [ 1.681513] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 1.685513] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 1.736545] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 1.740837] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored [ 1.745664] ACPI: Dynamic OEM Table Load: [ 1.777503] ACPI: Interpreter enabled [ 1.777524] ACPI: PM: (supports S0 S4 S5) [ 1.781514] ACPI: Using IOAPIC for interrupt routing [ 1.785536] HEST: Enabling Firmware First mode for corrected errors. [ 1.789592] HEST: Table parsing has been initialized. [ 1.793661] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. [ 1.797515] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 1.801513] PCI: Using E820 reservations for host bridge windows [ 1.806200] ACPI: Enabled 5 GPEs in block 00 to 3F [ 1.842831] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff]) [ 1.845516] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 1.851496] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 1.854566] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.857514] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.861544] PCI host bridge to bus 0000:ff [ 1.865514] pci_bus 0000:ff: root bus resource [bus ff] [ 1.869524] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000 [ 1.873579] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100 [ 1.877566] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100 [ 1.881565] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000 [ 1.885566] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000 [ 1.889568] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000 [ 1.893564] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000 [ 1.897565] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000 [ 1.901568] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000 [ 1.905564] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000 [ 1.909565] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000 [ 1.913564] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000 [ 1.917565] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000 [ 1.921566] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000 [ 1.925566] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100 [ 1.929566] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000 [ 1.933567] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100 [ 1.937564] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000 [ 1.941564] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000 [ 1.945539] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100 [ 1.949575] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000 [ 1.953615] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000 [ 1.957578] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000 [ 1.961577] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000 [ 1.965580] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000 [ 1.969577] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000 [ 1.973577] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000 [ 1.977661] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000 [ 1.981571] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000 [ 1.985580] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000 [ 1.989578] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000 [ 1.993579] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000 [ 1.997579] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000 [ 2.001566] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000 [ 2.005566] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000 [ 2.009566] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000 [ 2.013569] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000 [ 2.017579] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000 [ 2.021577] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000 [ 2.025579] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000 [ 2.029581] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000 [ 2.033571] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000 [ 2.037566] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000 [ 2.041568] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000 [ 2.045569] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000 [ 2.049567] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000 [ 2.053567] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000 [ 2.057567] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000 [ 2.061572] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000 [ 2.065568] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000 [ 2.069565] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000 [ 2.073566] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000 [ 2.077539] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000 [ 2.081567] pci 0000:ff:1f.0: [8086:6f88] type 00 class 0x088000 [ 2.085566] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000 [ 2.093968] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe]) [ 2.097516] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 2.101854] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 2.105986] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 2.109514] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration [ 2.113803] PCI host bridge to bus 0000:00 [ 2.117515] pci_bus 0000:00: root bus resource [bus 00-fe] [ 2.121513] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 2.125513] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window] [ 2.129513] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 2.133513] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window] [ 2.137513] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window] [ 2.141513] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window] [ 2.145620] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000 [ 2.149724] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400 [ 2.153579] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold [ 2.157689] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400 [ 2.161578] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold [ 2.165679] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400 [ 2.169578] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold [ 2.173674] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000 [ 2.177532] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit] [ 2.181708] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000 [ 2.185532] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit] [ 2.189705] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000 [ 2.193532] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit] [ 2.197706] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000 [ 2.201532] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit] [ 2.205704] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000 [ 2.209532] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit] [ 2.213710] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000 [ 2.217532] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit] [ 2.221705] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000 [ 2.225532] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit] [ 2.229705] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000 [ 2.233533] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit] [ 2.237658] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000 [ 2.241735] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000 [ 2.245661] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000 [ 2.249683] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020 [ 2.253528] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff] [ 2.257754] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000 [ 2.261806] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601 [ 2.265535] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117] [ 2.269525] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103] [ 2.273525] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7] [ 2.277525] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3] [ 2.281525] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f] [ 2.285525] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff] [ 2.289553] pci 0000:00:11.4: PME# supported from D3hot [ 2.293669] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330 [ 2.297540] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit] [ 2.301589] pci 0000:00:14.0: PME# supported from D3hot D3cold [ 2.305662] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000 [ 2.309539] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit] [ 2.313590] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold [ 2.317658] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000 [ 2.321539] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit] [ 2.325590] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold [ 2.329671] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320 [ 2.333536] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff] [ 2.337618] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold [ 2.341674] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400 [ 2.345603] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold [ 2.349676] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400 [ 2.353606] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold [ 2.357675] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400 [ 2.361605] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold [ 2.365684] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320 [ 2.369536] pci 0000:00:1d.0: reg 0x10: [mem 0xfb433000-0xfb4333ff] [ 2.373618] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold [ 2.377670] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100 [ 2.381789] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601 [ 2.385533] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077] [ 2.389524] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063] [ 2.393523] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057] [ 2.397523] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043] [ 2.401523] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f] [ 2.405524] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff] [ 2.409547] pci 0000:00:1f.2: PME# supported from D3hot [ 2.413657] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500 [ 2.417539] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit] [ 2.421545] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f] [ 2.425728] acpiphp: Slot [1] registered [ 2.429530] pci 0000:00:01.0: PCI bridge to [bus 01] [ 2.433692] pci 0000:02:00.0: [8086:0953] type 00 class 0x010802 [ 2.437534] pci 0000:02:00.0: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit] [ 2.441555] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref] [ 2.445516] pci 0000:02:00.0: enabling Extended Tags [ 2.449618] pci 0000:00:02.0: PCI bridge to [bus 02] [ 2.453515] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 2.457695] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000 [ 2.461535] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref] [ 2.465522] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f] [ 2.469536] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref] [ 2.473523] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref] [ 2.477554] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold [ 2.481534] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.485513] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 2.489526] pci 0000:03:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.493513] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs) [ 2.497676] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000 [ 2.501535] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref] [ 2.505522] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f] [ 2.509536] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref] [ 2.513522] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref] [ 2.517554] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold [ 2.521530] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.525513] pci 0000:03:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 2.529526] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.533514] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs) [ 2.537646] pci 0000:00:03.0: PCI bridge to [bus 03] [ 2.541514] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 2.545514] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 2.549515] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 2.553544] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 2.557579] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400 [ 2.561602] pci 0000:05:00.0: enabling Extended Tags [ 2.565586] pci 0000:05:00.0: supports D1 D2 [ 2.569513] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 2.573604] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 2.577515] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 2.581515] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 2.585561] pci_bus 0000:06: extended config space not accessible [ 2.589543] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000 [ 2.593547] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff] [ 2.597531] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff] [ 2.601530] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f] [ 2.605690] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 2.609556] pci 0000:06:00.0: supports D1 D2 [ 2.613513] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 2.617628] pci 0000:05:00.0: PCI bridge to [bus 06] [ 2.621519] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 2.625516] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 2.629612] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000 [ 2.633540] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff] [ 2.637541] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f] [ 2.641527] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff] [ 2.645649] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold [ 2.649552] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.653514] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 2.657539] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.661513] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 2.665690] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000 [ 2.669540] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff] [ 2.673541] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f] [ 2.677528] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff] [ 2.681644] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold [ 2.685551] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.689513] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 2.693539] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.697513] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 2.701670] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 2.705515] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 2.709515] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 2.713517] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08 [ 2.718436] ACPI: PCI: Interrupt link LNKA configured for IRQ 11 [ 2.721563] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 2.725562] ACPI: PCI: Interrupt link LNKC configured for IRQ 5 [ 2.729560] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 2.733563] ACPI: PCI: Interrupt link LNKE configured for IRQ 0 [ 2.737513] ACPI: PCI: Interrupt link LNKE disabled [ 2.741561] ACPI: PCI: Interrupt link LNKF configured for IRQ 0 [ 2.745513] ACPI: PCI: Interrupt link LNKF disabled [ 2.749561] ACPI: PCI: Interrupt link LNKG configured for IRQ 0 [ 2.753513] ACPI: PCI: Interrupt link LNKG disabled [ 2.757561] ACPI: PCI: Interrupt link LNKH configured for IRQ 0 [ 2.761513] ACPI: PCI: Interrupt link LNKH disabled [ 2.765701] iommu: Default domain type: Translated [ 2.769514] iommu: DMA domain TLB invalidation policy: lazy mode [ 2.773641] SCSI subsystem initialized [ 2.777522] pci 0000:06:00.0: vgaarb: setting as boot VGA device [ 2.781513] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none [ 2.781514] pci 0000:06:00.0: vgaarb: bridge control possible [ 2.785513] vgaarb: loaded [ 2.788301] ACPI: bus type USB registered [ 2.789523] usbcore: registered new interface driver usbfs [ 2.793517] usbcore: registered new interface driver hub [ 2.797518] usbcore: registered new device driver usb [ 2.801528] pps_core: LinuxPPS API ver. 1 registered [ 2.805513] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 2.809516] PTP clock support registered [ 2.813527] EDAC MC: Ver: 3.0.0 [ 2.816827] NetLabel: Initializing [ 2.817514] NetLabel: domain hash size = 128 [ 2.821513] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 2.825524] NetLabel: unlabeled traffic allowed by default [ 2.829522] PCI: Using ACPI for IRQ routing [ 2.838803] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0 [ 2.841514] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [ 2.847552] clocksource: Switched to clocksource tsc-early [ 2.859680] VFS: Disk quotas dquot_6.6.0 [ 2.863682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.870725] AppArmor: AppArmor Filesystem Enabled [ 2.875523] pnp: PnP ACPI init [ 2.879106] system 00:01: [io 0x0500-0x057f] has been reserved [ 2.885094] system 00:01: [io 0x0400-0x047f] has been reserved [ 2.891082] system 00:01: [io 0x0580-0x059f] has been reserved [ 2.897072] system 00:01: [io 0x0600-0x061f] has been reserved [ 2.903058] system 00:01: [io 0x0880-0x0883] has been reserved [ 2.909048] system 00:01: [io 0x0800-0x081f] has been reserved [ 2.915037] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved [ 2.921722] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved [ 2.928404] system 00:01: [mem 0xff000000-0xffffffff] has been reserved [ 2.935086] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved [ 2.941769] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved [ 2.948451] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved [ 2.955135] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved [ 2.962028] system 00:02: [io 0x0a00-0x0a0f] has been reserved [ 2.968014] system 00:02: [io 0x0a10-0x0a1f] has been reserved [ 2.974001] system 00:02: [io 0x0a20-0x0a2f] has been reserved [ 2.979992] system 00:02: [io 0x0a30-0x0a3f] has been reserved [ 2.985982] system 00:02: [io 0x0a40-0x0a4f] has been reserved [ 2.992949] pnp: PnP ACPI: found 5 devices [ 3.002500] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 3.011503] NET: Registered PF_INET protocol family [ 3.016611] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 3.025904] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, linear) [ 3.034728] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 3.042724] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 3.051130] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear) [ 3.058663] TCP: Hash tables configured (established 262144 bind 65536) [ 3.065475] MPTCP token hash table entries: 32768 (order: 7, 786432 bytes, linear) [ 3.073252] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 3.080270] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 3.087727] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 3.093462] NET: Registered PF_XDP protocol family [ 3.098340] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000 [ 3.109915] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref] [ 3.117828] pci 0000:00:01.0: PCI bridge to [bus 01] [ 3.122872] pci 0000:00:02.0: PCI bridge to [bus 02] [ 3.127906] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 3.134772] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.142424] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.150426] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.158164] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.166254] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.173906] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.181907] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.189645] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.197735] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.205384] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.213386] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.221124] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.229213] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.236865] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.244868] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.252606] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.260695] pci 0000:00:03.0: PCI bridge to [bus 03] [ 3.265722] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 3.271883] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 3.278740] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 3.286573] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 3.291622] pci 0000:05:00.0: PCI bridge to [bus 06] [ 3.296649] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 3.302812] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 3.309686] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 3.314980] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 3.321142] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 3.328006] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref] [ 3.335835] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref] [ 3.343758] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref] [ 3.351593] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref] [ 3.359515] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 3.364549] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 3.370710] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 3.377566] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref] [ 3.385402] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 3.391646] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window] [ 3.397897] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 3.412523] pci_bus 0000:00: resource 7 [mem 0x90000000-0xfbffbfff window] [ 3.419461] pci_bus 0000:00: resource 8 [mem 0xfedb0000-0xfedb000f window] [ 3.426404] pci_bus 0000:00: resource 9 [mem 0xfedc0000-0xfedc000f window] [ 3.433347] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff] [ 3.439681] pci_bus 0000:03: resource 0 [io 0xe000-0xefff] [ 3.445322] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff] [ 3.451651] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref] [ 3.458961] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff] [ 3.464599] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff] [ 3.470935] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff] [ 3.476580] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff] [ 3.482915] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff] [ 3.488555] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff] [ 3.494884] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref] [ 3.502508] pci 0000:00:05.0: disabled boot interrupts on device [8086:6f28] [ 3.531824] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x160 took 21521 usecs [ 3.563809] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x160 took 23914 usecs [ 3.571310] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes [ 3.577861] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.577870] Trying to unpack rootfs image as initramfs... [ 3.584368] software IO TLB: mapped [mem 0x0000000074ec2000-0x0000000078ec2000] (64MB) [ 3.598308] Initialise system trusted keyrings [ 3.602830] Key type blacklist registered [ 3.606950] workingset: timestamp_bits=36 max_order=23 bucket_order=0 [ 3.614419] zbud: loaded [ 3.617228] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 3.623289] fuse: init (API version 7.34) [ 3.627512] integrity: Platform Keyring initialized [ 3.640148] Key type asymmetric registered [ 3.644316] Asymmetric key parser 'x509' registered [ 3.649284] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243) [ 3.656809] io scheduler mq-deadline registered [ 3.661877] pcieport 0000:00:01.0: PME: Signaling with IRQ 25 [ 3.667912] pcieport 0000:00:02.0: PME: Signaling with IRQ 27 [ 3.673935] pcieport 0000:00:03.0: PME: Signaling with IRQ 29 [ 3.679910] pcieport 0000:00:1c.0: PME: Signaling with IRQ 30 [ 3.685889] pcieport 0000:00:1c.2: PME: Signaling with IRQ 31 [ 3.691850] pcieport 0000:00:1c.4: PME: Signaling with IRQ 32 [ 3.697813] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.704716] ACPI: \_SB_.SCK0.CP00: Found 2 idle states [ 3.710461] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0 [ 3.718936] ACPI: button: Power Button [PWRB] [ 3.723395] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 [ 3.730911] ACPI: button: Power Button [PWRF] [ 3.736494] ERST: Error Record Serialization Table (ERST) support is initialized. [ 3.744068] pstore: Registered erst as persistent store backend [ 3.750184] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled [ 3.777121] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.806029] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.815613] Linux agpgart interface v0.103 [ 3.821539] loop: module loaded [ 3.824974] tun: Universal TUN/TAP device driver, 1.6 [ 3.830116] PPP generic driver version 2.4.2 [ 3.834497] VFIO - User Level meta-driver version: 0.3 [ 3.839793] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 3.846393] ehci-pci: EHCI PCI platform driver [ 3.851056] ehci-pci 0000:00:1a.0: EHCI Host Controller [ 3.856359] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1 [ 3.863856] ehci-pci 0000:00:1a.0: debug port 2 [ 3.872385] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000 [ 3.891703] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00 [ 3.897560] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15 [ 3.905915] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.913230] usb usb1: Product: EHCI Host Controller [ 3.918179] usb usb1: Manufacturer: Linux 5.15.0-102-generic ehci_hcd [ 3.924693] usb usb1: SerialNumber: 0000:00:1a.0 [ 3.929521] hub 1-0:1.0: USB hub found [ 3.931689] Freeing initrd memory: 65088K [ 3.933347] hub 1-0:1.0: 2 ports detected [ 3.941752] ehci-pci 0000:00:1d.0: EHCI Host Controller [ 3.947046] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2 [ 3.954538] ehci-pci 0000:00:1d.0: debug port 2 [ 3.963020] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000 [ 3.983738] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00 [ 3.989590] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15 [ 3.997945] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 4.005248] usb usb2: Product: EHCI Host Controller [ 4.010190] usb usb2: Manufacturer: Linux 5.15.0-102-generic ehci_hcd [ 4.016699] usb usb2: SerialNumber: 0000:00:1d.0 [ 4.021490] hub 2-0:1.0: USB hub found [ 4.025314] hub 2-0:1.0: 2 ports detected [ 4.029505] ehci-platform: EHCI generic platform driver [ 4.034816] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 4.041062] ohci-pci: OHCI PCI platform driver [ 4.045591] ohci-platform: OHCI generic platform driver [ 4.050882] uhci_hcd: USB Universal Host Controller Interface driver [ 4.057350] i8042: PNP: No PS/2 controller found. [ 4.062185] mousedev: PS/2 mouse device common for all mice [ 4.067898] rtc_cmos 00:00: RTC can wake from S4 [ 4.072803] rtc_cmos 00:00: registered as rtc0 [ 4.077342] rtc_cmos 00:00: setting system clock to 2024-04-18T22:25:42 UTC (1713479142) [ 4.085545] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram [ 4.092318] i2c_dev: i2c /dev entries driver [ 4.096828] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. [ 4.109109] device-mapper: uevent: version 1.0.3 [ 4.113858] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com [ 4.122402] platform eisa.0: Probing EISA bus 0 [ 4.127003] platform eisa.0: EISA: Cannot allocate resource for mainboard [ 4.133855] platform eisa.0: Cannot allocate resource for EISA slot 1 [ 4.140363] platform eisa.0: Cannot allocate resource for EISA slot 2 [ 4.146874] platform eisa.0: Cannot allocate resource for EISA slot 3 [ 4.153382] platform eisa.0: Cannot allocate resource for EISA slot 4 [ 4.159891] platform eisa.0: Cannot allocate resource for EISA slot 5 [ 4.166402] platform eisa.0: Cannot allocate resource for EISA slot 6 [ 4.172910] platform eisa.0: Cannot allocate resource for EISA slot 7 [ 4.179420] platform eisa.0: Cannot allocate resource for EISA slot 8 [ 4.185929] platform eisa.0: EISA: Detected 0 cards [ 4.190880] intel_pstate: Intel P-state driver initializing [ 4.197460] ledtrig-cpu: registered to indicate activity on CPUs [ 4.203681] drop_monitor: Initializing network drop monitor service [ 4.210126] NET: Registered PF_INET6 protocol family [ 4.222449] Segment Routing with IPv6 [ 4.226210] In-situ OAM (IOAM) with IPv6 [ 4.230212] NET: Registered PF_PACKET protocol family [ 4.235434] Key type dns_resolver registered [ 4.240190] microcode: sig=0x406f1, pf=0x1, revision=0xb000040 [ 4.246164] microcode: Microcode Update Driver: v2.2. [ 4.246242] resctrl: L3 allocation detected [ 4.255600] resctrl: L3 monitoring detected [ 4.259852] IPI shorthand broadcast: enabled [ 4.264207] sched_clock: Marking stable (3216524796, 1047665431)->(5253335923, -989145696) [ 4.272784] registered taskstats version 1 [ 4.275708] usb 1-1: new high-speed USB device number 2 using ehci-pci [ 4.283668] Loading compiled-in X.509 certificates [ 4.289099] Loaded X.509 cert 'Build time autogenerated kernel key: 8dbd8aa72291a99043f1138db79927b696791b59' [ 4.299531] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969' [ 4.309765] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19' [ 4.319852] blacklist: Loading compiled-in revocation X.509 certificates [ 4.326632] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0' [ 4.336552] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03' [ 4.347087] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b' [ 4.357961] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8' [ 4.367726] usb 2-1: new high-speed USB device number 2 using ehci-pci [ 4.368495] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d' [ 4.385878] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c' [ 4.396675] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af' [ 4.407469] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9' [ 4.419292] zswap: loaded using pool lzo/zbud [ 4.424023] Key type .fscrypt registered [ 4.428032] Key type fscrypt-provisioning registered [ 4.433176] pstore: Using crash dump compression: deflate [ 4.441752] Key type encrypted registered [ 4.445833] AppArmor: AppArmor sha1 policy hashing enabled [ 4.451397] ima: No TPM chip found, activating TPM-bypass! [ 4.456954] Loading compiled-in module X.509 certificates [ 4.457021] usb 1-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05 [ 4.462877] Loaded X.509 cert 'Build time autogenerated kernel key: 8dbd8aa72291a99043f1138db79927b696791b59' [ 4.470697] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 4.488479] ima: Allocated hash algorithm: sha1 [ 4.500295] ima: No architecture policies found [ 4.500373] hub 1-1:1.0: USB hub found [ 4.504912] evm: Initialising EVM extended attributes: [ 4.513942] evm: security.selinux [ 4.517330] evm: security.SMACK64 [ 4.520719] evm: security.SMACK64EXEC [ 4.524455] evm: security.SMACK64TRANSMUTE [ 4.524501] hub 1-1:1.0: 6 ports detected [ 4.528617] evm: security.SMACK64MMAP [ 4.528617] evm: security.apparmor [ 4.528619] evm: security.ima [ 4.528619] evm: security.capability [ 4.528620] evm: HMAC attrs: 0x1 [ 4.528881] PM: Magic number: 4:51:452 [ 4.554291] RAS: Correctable Errors collector initialized. [ 4.560076] usb 2-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05 [ 4.568350] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 4.575835] Freeing unused decrypted memory: 2036K [ 4.575847] hub 2-1:1.0: USB hub found [ 4.584576] hub 2-1:1.0: 8 ports detected [ 4.584906] Freeing unused kernel image (initmem) memory: 3352K [ 4.599686] Write protecting the kernel read-only data: 30720k [ 4.605971] Freeing unused kernel image (text/rodata gap) memory: 2036K [ 4.612887] Freeing unused kernel image (rodata/data gap) memory: 1340K [ 4.627720] tsc: Refined TSC clocksource calibration: 3499.996 MHz [ 4.633997] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734d20948, max_idle_ns: 440795306379 ns [ 4.662881] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 4.669392] x86/mm: Checking user space page tables [ 4.674348] clocksource: Switched to clocksource tsc [ 4.711609] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 4.718119] Run /init as init process Loading, please wait... Starting version 249.11-0ubuntu3.6 [ 4.885819] dca service started, version 1.12.1 [ 4.891909] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 4.898318] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3 [ 4.906889] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810 [ 4.907040] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode [ 4.916603] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 4.924300] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst [ 4.937414] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4 [ 4.937490] igb: Intel(R) Gigabit Ethernet Network Driver [ 4.944910] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed [ 4.950371] igb: Copyright (c) 2007-2014 Intel Corporation. [ 4.962384] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15 [ 4.963967] scsi host0: ahci [ 4.970746] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 4.973812] scsi host1: ahci [ 4.981004] usb usb3: Product: xHCI Host Controller [ 4.984054] scsi host2: ahci [ 4.988898] usb usb3: Manufacturer: Linux 5.15.0-102-generic xhci-hcd [ 4.991921] scsi host3: ahci [ 4.998354] usb usb3: SerialNumber: 0000:00:14.0 [ 5.001344] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 33 [ 5.004236] igb 0000:07:00.0: added PHC on eth0 [ 5.004260] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection [ 5.004261] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:bc [ 5.004338] igb 0000:07:00.0: eth0: PBA No: 050B00-000 [ 5.004339] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 5.006144] hub 3-0:1.0: USB hub found [ 5.013494] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 33 [ 5.018132] hub 3-0:1.0: 15 ports detected [ 5.025041] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 33 [ 5.034628] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.15 [ 5.037447] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 33 [ 5.045106] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 5.049066] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver [ 5.049181] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode [ 5.049184] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst [ 5.056416] usb usb4: Product: xHCI Host Controller [ 5.056418] usb usb4: Manufacturer: Linux 5.15.0-102-generic xhci-hcd [ 5.056420] usb usb4: SerialNumber: 0000:00:14.0 [ 5.058348] igb 0000:07:00.1: added PHC on eth1 [ 5.058375] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection [ 5.058377] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:bd [ 5.058452] igb 0000:07:00.1: eth1: PBA No: 050B00-000 [ 5.058454] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 5.060584] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 5.068168] hub 4-0:1.0: USB hub found [ 5.112200] scsi host4: ahci [ 5.113312] hub 4-0:1.0: 6 ports detected [ 5.118338] scsi host5: ahci [ 5.141066] nvme nvme0: pci function 0000:02:00.0 [ 5.141124] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt [ 5.142101] i2c i2c-2: 2/8 memory slots populated (from DMI) [ 5.142103] i2c i2c-2: Systems with more than 4 memory slots not supported yet, not instantiating SPD [ 5.142126] igb 0000:07:00.0 eno1: renamed from eth0 [ 5.148303] scsi host6: ahci [ 5.156228] nvme nvme0: 8/0/0 default/read/poll queues [ 5.161169] scsi host7: ahci [ 5.222532] scsi host8: ahci [ 5.225603] scsi host9: ahci [ 5.228574] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 54 [ 5.236056] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 54 [ 5.24354ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 54 [ 5.251021] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 54 [ 5.258500] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 54 [ 5.265980] ata10: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432380 irq 54 [ 5.362994] ata4: SATA link down (SStatus 0 SControl 300) [ 5.368502] ata1: SATA link down (SStatus 0 SControl 300) [ 5.374000] ata2: SATA link down (SStatus 0 SControl 300) [ 5.375671] usb 3-12: new high-speed USB device number 2 using xhci_hcd [ 5.379500] ata3: SATA link down (SStatus 0 SControl 300) [ 5.531937] usb 3-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00 [ 5.540314] usb 3-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 5.548043] hub 3-12:1.0: USB hub found [ 5.552035] hub 3-12:1.0: 4 ports detected [ 5.586137] ata8: SATA link down (SStatus 0 SControl 300) [ 5.591643] ata9: SATA link down (SStatus 0 SControl 300) [ 5.597150] ata6: SATA link down (SStatus 0 SControl 300) [ 5.602643] ata7: SATA link down (SStatus 0 SControl 300) [ 5.608134] ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 5.614413] ata10: SATA link down (SStatus 0 SControl 300) [ 5.620160] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133 [ 5.626996] ata5.00: NCQ Send/Recv Log not supported [ 5.632056] ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 32), AA [ 5.640462] ata5.00: NCQ Send/Recv Log not supported [ 5.645518] ata5.00: configured for UDMA/133 [ 5.650093] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5 [ 5.658555] sd 4:0:0:0: Attached scsi generic sg0 type 0 [ 5.658733] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB) [ 5.671724] sd 4:0:0:0: [sda] Write Protect is off [ 5.676602] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 5.700408] sda: sda1 [ 5.703094] sd 4:0:0:0: [sda] Attached SCSI disk [ 5.709366] igb 0000:07:00.1 eno2: renamed from eth1 [ 5.879725] usb 3-12.1: new low-speed USB device number 3 using xhci_hcd [ 6.012173] usb 3-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00 [ 6.020722] usb 3-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 6.039518] hid: raw HID events driver (C) Jiri Kosina [ 6.047520] usbcore: registered new interface driver usbhid [ 6.053184] usbhid: USB HID core driver [ 6.059675] i HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.0/0003:0557:2419.0001/input/input2 [ 6.127897] hid-generic 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0 [ 6.139959] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.1/0003:0557:2419.0002/input/input3 [ 6.151886] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1 [ 6.264561] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 6.274962] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 6.283584] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF [ 6.289839] ixgbe 0000:03:00.0: 0c:c4:7a:8f:cc:0c [ 6.306223] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection [ 6.476499] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 6.486902] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 6.495526] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF [ 6.502656] ixgbe 0000:03:00.1: 0c:c4:7a:8f:cc:0d [ 6.511018] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection [ 6.519309] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1 [ 6.540281] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done. Begin: Will now check root file system ... fsck from util-linux 2.37.2 [/usr/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1 /dev/sda1: recovering journal /dev/sda1: Clearing orphaned inode 24691 (uid=0, gid=0, mode=0100644, size=4447536) /dev/sda1: clean, 198140/61054976 files, 6688498/244190336 blocks done. [ 13.680307] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none. done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... done. [ 14.505308] systemd[1]: Inserted module 'autofs4' [ 14.602885] systemd[1]: systemd 249.11-0ubuntu3.6 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 14.651797] systemd[1]: Detected architecture x86-64. Welcome to Ubuntu 22.04.1 LTS! [ 14.719401] systemd[1]: Hostname set to . [ 16.114444] systemd[1]: Queued start job for default target Graphical Interface. [ 16.124619] systemd[1]: Created slice Slice /system/modprobe. [ OK ] Created slice Slice /system/modprobe. [ 16.148624] systemd[1]: Created slice Slice /system/serial-getty. [ OK ] Created slice Slice /system/serial-getty. [ 16.172482] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ 16.195879] systemd[1]: Started ntp-systemd-netif.path. [ OK ] Started ntp-systemd-netif.path. [ 16.215925] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password R…uests to Wall Directory Watch. [ 16.240127] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. [ OK ] Set up automount Arbitrary…s File System Automount Point. [ 16.267865] systemd[1]: Reached target ceph target allowing to start/stop all ceph-fuse@.service instances at once. [ OK ] Reached target ceph target…se@.service instances at once. [ 16.295822] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mon@.service instances at once. [ OK ] Reached target ceph target…on@.service instances at once. [ 16.323818] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mds@.service instances at once. [ OK ] Reached target ceph target…ds@.service instances at once. [ 16.351840] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mgr@.service instances at once. [ OK ] Reached target ceph target…gr@.service instances at once. [ 16.379809] systemd[1]: Reached target ceph target allowing to start/stop all ceph-osd@.service instances at once. [ OK ] Reached target ceph target…sd@.service instances at once. [ 16.407805] systemd[1]: Reached target ceph target allowing to start/stop all ceph-radosgw@.service instances at once. [ OK ] Reached target ceph target…gw@.service instances at once. [ 16.435901] systemd[1]: Reached target User and Group Name Lookups. [ OK ] Reached target User and Group Name Lookups. [ 16.459839] systemd[1]: Reached target Slice Units. [ OK ] Reached target Slice Units. [ 16.479878] systemd[1]: Reached target Local Verity Protected Volumes. [ OK ] Reached target Local Verity Protected Volumes. [ 16.503940] systemd[1]: Listening on Device-mapper event daemon FIFOs. [ OK ] Listening on Device-mapper event daemon FIFOs. [ 16.528060] systemd[1]: Listening on LVM2 poll daemon socket. [ OK ] Listening on LVM2 poll daemon socket. [ 16.554526] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ 16.580079] systemd[1]: Listening on Syslog Socket. [ OK ] Listening on Syslog Socket. [ 16.600048] systemd[1]: Listening on fsck to fsckd communication Socket. [ OK ] Listening on fsck to fsckd communication Socket. [ 16.623964] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ 16.648241] systemd[1]: Listening on Journal Audit Socket. [ OK ] Listening on Journal Audit Socket. [ 16.668037] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 16.692076] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 16.712187] systemd[1]: Listening on Network Service Netlink Socket. [ OK ] Listening on Network Service Netlink Socket. [ 16.736099] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 16.756013] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 16.777513] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ 16.797552] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... [ 16.821262] systemd[1]: Mounting NFSD configuration filesystem... Mounting NFSD configuration filesystem... [ 16.845451] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ 16.865586] systemd[1]: Mounting Kernel Trace File System... Mounting Kernel Trace File System... [ 16.880738] RPC: Registered named UNIX socket transport module. [ 16.886737] RPC: Registered udp transport module. [ 16.891515] RPC: Registered tcp transport module. [ 16.896293] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 16.904181] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.923819] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped. [ 16.934202] systemd[1]: Starting Set the console keyboard layout... Starting Set the console keyboard layout... [ 16.956844] systemd[1]: Starting Create List of Static Device Nodes... Starting Create List of Static Device Nodes... [ 16.980436] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... Starting Monitoring of LVM…meventd or progress polling... [ 17.008848] systemd[1]: Starting Load Kernel Module chromeos_pstore... Starting Load Kernel Module chromeos_pstore... [ 17.032512] systemd[1]: Starting Load Kernel Module configfs... Starting Load Kernel Module configfs... [ 17.052484] systemd[1]: Starting Load Kernel Module drm... Starting Load Kernel Module drm... [ 17.072485] systemd[1]: Starting Load Kernel Module efi_pstore... Starting Load Kernel Module efi_pstore... [ 17.096877] systemd[1]: Starting Load Kernel Module fuse... [ 17.097730] Installing knfsd (copyright (C) 1996 okir@monad.swb.de). Starting Load Kernel Module fuse... [ 17.125285] systemd[1]: Starting Load Kernel Module pstore_blk... Starting Load Kernel Module pstore_blk... [ 17.149185] systemd[1]: Starting Load Kernel Module pstore_zone... Starting Load Kernel Module pstore_zone... [ 17.173153] systemd[1]: Starting Load Kernel Module ramoops... Starting Load Kernel Module ramoops... [ 17.191855] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped. [ 17.201709] systemd[1]: Condition check resulted in File System Check on Root Device being skipped. [ 17.212800] systemd[1]: Starting Load Kernel Modules... Starting Load Kernel Modules... [ 17.233096] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ 17.256614] systemd[1]: Starting Coldplug All udev Devices... Starting Coldplug All udev Devices... [ 17.277551] systemd[1]: Started Journal Service. [ OK ] Started Journal Service. [ 17.286907] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro,user_xattr. Quota mode: none. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ 17.323552] lp: driver loaded but no devices found [ OK ] Mounted NFSD configuration filesystem. [ OK ] Mounted Kernel Debug File System. [ 17.357248] Loading iSCSI transport class v2.0-870. [ OK ] Mounted Kernel Trace File System. [ OK ] Finished Set the console keyboard layout. [ OK ] Finished Create List of Static Device Nodes. [ OK ] Finished Monitoring of LVM… dmeventd or progress polling. [ OK ] Finished Load Kernel Module chromeos_pstore. [ OK ] Finished Load Kernel Module configfs. [ OK ] Finished Load Kernel Module drm. [ OK ] Finished Load Kernel Module efi_pstore. [ OK ] Finished Load Kernel Module fuse. [ OK ] Finished Load Kernel Module pstore_blk. [ OK ] Finished Load Kernel Module pstore_zone. [ OK ] Finished Load Kernel Module ramoops. [ OK ] Finished Load Kernel Modules. [ OK ] Finished Remount Root and Kernel File Systems. Activating swap /swapfile... Mounting FUSE Control File System... Mounting Kernel Configuration File System... Starting Initial Check File System Quotas... Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Apply Kernel Variables... [ 19.431026] Adding 2097148k swap on /swapfile. Priority:-2 extents:6 across:2260988k FS Starting Create System Users[[ 19.444842] systemd-journald[366]: Received client request to flush runtime journal. 0m... [ OK ] Activated swap /swapfile. [ OK ] Finished Coldplug All udev Devices. [ OK ] Mounted FUSE Control File System. [ OK ] Mounted Kernel Configuration File System. [ OK ] Reached target Swaps. Starting Helper to synchronize boot up for ifupdown... [ OK ] Finished Load/Save Random Seed. [ 19.705641] systemd-journald[366]: File /var/log/journal/5896a171b9d64a25b2db7eb12a64f597/system.journal corrupted or uncleanly shut down, renaming and replacing. [ OK ] Finished Create System Users. Starting Create Static Device Nodes in /dev... [ OK ] Finished Apply Kernel Variables. [ OK ] Finished Initial Check File System Quotas. [ OK ] Finished Flush Journal to Persistent Storage. [ OK ] Finished Create Static Device Nodes in /dev. [ OK ] Reached target Preparation for Local File Systems. [ OK ] Reached target Local File Systems. Starting Load AppArmor profiles... Starting Set console font and keymap... Starting Create final runt…dir for shutdown pivot root... Starting Tell Plymouth To Write Out Runtime Data... Starting QEMU KVM preparation - module, ksm, hugepages... Starting Create Volatile Files and Directories... Starting Rule-based Manage…for Device Events and Files... Starting Uncomplicated firewall... [ OK ] Finished Set console font and keymap. [ OK ] Finished Create final runt…e dir for shutdown pivot root. [ OK ] Finished Tell Plymouth To Write Out Runtime Data. [ OK ] Finished Uncomplicated firewall. [ OK ] Reached target Preparation for Network. [ OK ] Finished Create Volatile Files and Directories. Mounting RPC Pipe File System... Starting Load Kernel Module chromeos_pstore... Starting Load Kernel Module efi_pstore... Starting Load Kernel Module pstore_blk... Starting Load Kernel Module pstore_zone... Starting Load Kernel Module ramoops... Starting RPC bind portmap service... Starting Record System Boot/Shutdown in UTMP... [ OK ] Finished Load AppArmor profiles. [ OK ] Started Rule-based Manager for Device Events and Files. [ OK ] Mounted RPC Pipe File System. [ OK ] Finished Load Kernel Module chromeos_pstore. [ OK ] Finished Load Kernel Module efi_pstore. [ OK ] Finished QEMU KVM preparation - module, ksm, hugepages. [ OK ] Finished Load Kernel Module pstore_blk. [ OK ] Finished Load Kernel Module pstore_zone. [ OK ] Finished Load Kernel Module ramoops. [ OK ] Reached target rpc_pipefs.target. Starting pNFS block layout mapping daemon... Starting NFSv4 ID-name mapping service... Starting NFSv4 Client Tracking Daemon... [ OK ] Started Dispatch Password …ts to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Path Units. [ OK ] Reached target NFS client services. Starting Network Configuration... [ OK ] Started RPC bind portmap service. [ OK ] Finished Record System Boot/Shutdown in UTMP. [ OK ] Started pNFS block layout mapping daemon. [ OK ] Started NFSv4 ID-name mapping service. [ OK ] Started NFSv4 Client Tracking Daemon. [ OK ] Found device /dev/ttyS1. [ OK ] Found device /dev/vg_nvme/lv_5. [ OK ] Created slice Slice /system/lvm2-pvscan. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target System Initialization. [ OK ] Started Daily apt download activities. [ OK ] Started Daily apt upgrade and clean activities. [ OK ] Started Daily dpkg database backup timer. [ OK ] Started Periodic ext4 Onli…ata Check for All Filesystems. [ OK ] Started Discard unused blocks once a week. [ OK ] Started Daily rotation of log files. [ OK ] Started Daily man-db regeneration. [ OK ] Started Message of the Day. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started Ubuntu Advantage Timer for running repeated jobs. [ OK ] Reached target Timer Units. [ OK ] Listening on D-Bus System Message Bus Socket. Starting Docker Socket for the API... [ OK ] Listening on Open-iSCSI iscsid Socket. [ OK ] Listening on Load/Save RF …itch Status /dev/rfkill Watch. [ OK ] Listening on UUID daemon activation socket. Mounting /var/lib/ceph... Starting LVM event activation on device 259:0... [ OK ] Started Network Configuration. [ OK ] Finished Helper to synchronize boot up for ifupdown. [ OK ] Listening on Docker Socket for the API. [ OK ] Mounted /var/lib/ceph. [ OK ] Found device 82599ES 10-Gi…ork Connection (AOC-STGN-i2S). [ OK ] Finished LVM event activation on device 259:0. [ OK ] Reached target Socket Units. [ OK ] Reached target Basic System. Starting Accounts Service... [ OK ] Started Ceph crash dump collector. [ OK ] Reached target ceph target…h*@.service instances at once. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Save initial kernel messages after boot. Starting Remove Stale Onli…t4 Metadata Check Snapshots... Starting Record successful boot for GRUB... [ OK ] Started 1. [ OK ] Started irqbalance daemon. [ OK ] Started LTTng session daemon. Starting Dispatcher daemon for systemd-networkd... Starting Raise network interfaces... [ OK ] Started ntp-systemd-netif.service. Starting RPC Remote Quota Server... Starting System Logging Service... Starting Self Monitoring a…g Technology (SMART) Daemon... Starting User Login Management... Starting Wait for Network to be Configured... Starting Network Name Resolution... [ OK ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots. [ OK ] Started RPC Remote Quota Server. [ OK ] Finished Record successful boot for GRUB. [ OK ] Started User Login Management. Starting GRUB failed boot detection... [ OK ] Started System Logging Service. [ OK ] Finished GRUB failed boot detection. [ OK ] Started Self Monitoring an…ing Technology (SMART) Daemon. [ OK ] Started Network Name Resolution. [ OK ] Reached target Host and Network Name Lookups. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Accounts Service. [ OK ] Started Dispatcher daemon for systemd-networkd. [* ] (1 of 2) A start job is running for… network interfaces (6s / 5min 1s) M [ OK ] Finished Raise network interfaces. [ OK ] Reached target Network. Starting containerd container runtime... Starting Network Time Service... Starting OpenBSD Secure Shell server... Starting (i)SCSI target daemon... [ OK ] Started Unattended Upgrades Shutdown. [ OK ] Started Network Time Service. [ OK ] Started OpenBSD Secure Shell server. [ OK ] Started (i)SCSI target daemon. [ OK ] Started containerd container runtime. [ OK ] Started ntp-systemd-netif.service. [ OK ] Finished Wait for Network to be Configured. [ OK ] Reached target Network is Online. Starting Docker Application Container Engine... Starting NFS Mount Daemon... Starting Connect NVMe-oF s…s automatically during boot... Starting Map RBD devices... Starting /etc/rc.local Compatibility... + '[' '!' -f /.cephlab_net_configured ']' + set +e + attempts=0 + myips= + '[' '' '!=' '' ']' + '[' 0 -ge 10 ']' ++ grep -oP '(?<=inet\s)\d+(\.\d+){3}' ++ ip -4 addr ++ grep -v '127.0.0.1\|127.0.1.1' Starting NFS status monitor for NFSv2/3 locking.... Starting Ubuntu FAN network setup... [ OK ] Finished Map RBD devices. [ OK ] Finished Availability of block devices. [ OK ] Started NFS Mount Daemon. [ OK ] Started NFS status monitor for NFSv2/3 locking.. Starting NFS server and services... + myips=172.21.15.96 + attempts=1 + sleep 1 [ OK ] Finished Connect NVMe-oF s…ems automatically during boot. [ OK ] Reached target Preparation for Remote File Systems. [ OK ] Reached target Remote File Systems. Starting The Apache HTTP Server... Starting LSB: Collectl monitors system performance.... [ OK ] Started Regular background program processing daemon. [ OK ] Started Nagios Remote Plugin Executor. Starting LSB: radosgw RESTful rados gateway... Starting Permit User Sessions... [ OK ] Finished Permit User Sessions. [ OK ] Started LSB: radosgw RESTful rados gateway. [ OK ] Finished Ubuntu FAN network setup. [ OK ] Finished NFS server and services. Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. + '[' 172.21.15.96 '!=' '' ']' + set -e + '[' -n 172.21.15.96 ']' + for ip in $myips + timeout 1s ping -I 172.21.15.96 -nq -c1 172.21.0.1 ++ dig +short -x 172.21.15.96 @172.21.0.1 ++ sed 's/\.com.*/\.com/g' [ OK ] Started The Apache HTTP Server. + newhostname=smithi096.front.sepia.ceph.com + '[' -n smithi096.front.sepia.ceph.com ']' + hostname smithi096.front.sepia.ceph.com ++ hostname -d + newdomain=front.sepia.ceph.com ++ hostname -s + shorthostname=smithi096 + echo smithi096 + grep -q front.sepia.ceph.com /etc/hosts + sed -i 's/.*front.sepia.ceph.com.*/172.21.15.96 smithi096.front.sepia.ceph.com smithi096/g' /etc/hosts + break + command -v zypper + command -v apt-get + '[' '!' -f /etc/ssh/ssh_host_rsa_key ']' + '[' -e /.cephlab_rc_local ']' + exit 0 [ OK ] Started /etc/rc.local Compatibility. Starting Hold until boot process finishes up... Starting Terminate Plymouth Boot Screen... [ OK ] Finished Hold until boot process finishes up. [ OK ] Finished Terminate Plymouth Boot Screen. [ OK ] Started Serial Getty on ttyS1. Starting Set console scheme... [ OK ] Finished Set console scheme. [ OK ] Created slice Slice /system/getty. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started LSB: Collectl monitors system performance.. [ OK ] Started Docker Application Container Engine. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Record Runlevel Change in UTMP... [ OK ] Finished Record Runlevel Change in UTMP. Ubuntu 22.04.1 LTS smithi096.front.sepia.ceph.com ttyS1 2024-04-18T22:26:14.602 DEBUG:teuthology.orchestra.console.smithi096:expect after: smithi096 login: 2024-04-18T22:26:14.654 DEBUG:teuthology.orchestra.console.smithi096:console disconnect output: [Enter `^Ec?' for help] \x80 \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent GE v1.5.13 PXE 2.1 Build 092 (WfM 2.0) Press Ctrl+S to enter the Setup Menu.                       Press Ctrl+S to enter the Setup Menu..  \x80 \x80 \x80 \x80  Intel(R) Boot Agent XE v2.1.60 Copyright (C) 1997-2009, Intel Corporation  CLIENT MAC ADDR: 0C C4 7A 8F CC 0D GUID: 00000000 0000 0000 0000 0CC47AD8D9BC DHCP./                    DHCP.| DHCP./ DHCP.- DHCP.\ DHCP.| DHCP./ CLIENT IP: 172.21.15.96 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 GATEWAY IP: 172.21.15.254 PXE->EB: !PXE at 96D3:0070, entry point at 96D3:0106  UNDI code segment 96D3:3E70, data segment 8F94:73F0 (574-619kB)  UNDI device is PCI 03:00.1, type DIX+802.3  574kB free base memory after PXE unload iPXE initialising devices... iPXE initialising devices...ok iPXE 1.20.1+ (g4bd0) -- Open Source Network Boot Firmware -- http://ipxe.org Features: DNS FTP HTTP HTTPS iSCSI NFS TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT Configuring (net0 0c:c4:7a:8f:cc:0d)... Configuring (net0 0c:c4:7a:8f:cc:0d).... Configuring (net0 0c:c4:7a:8f:cc:0d)..... Configuring (net0 0c:c4:7a:8f:cc:0d)...... ok Received DHCP answer on interface net0  GNU GRUB version 2.04   ******************************************************************************  * Ubuntu *  **Advanced options for Ubuntu *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  ******************************************************************************   Use the * and * keys to select which entry is highlighted.  Press enter to boot the selected OS, `e' to edit the commands  before booting or `c' for a command-line.  The highlighted entry will be executed automatically in 4s.  The highlighted entry will be executed automatically in 3s.  The highlighted entry will be executed automatically in 2s.  The highlighted entry will be executed automatically in 1s.                     Loading Linux 5.15.0-102-generic ... Loading initial ramdisk ... [ 0.000000] microcode: microcode updated early to revision 0xb000040, date = 2021-05-19 [ 0.000000] Linux version 5.15.0-102-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 (Ubuntu 5.15.0-102.112-generic 5.15.148) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic root=UUID=7e9e2103-8bb7-4452-b35a-3b1230743254 ro pci=realloc=off console=tty0 console=ttyS1,115200 [ 0.000000] KERNEL supported cpus: [ 0.000000] Intel GenuineIntel [ 0.000000] AMD AuthenticAMD [ 0.000000] Hygon HygonGenuine [ 0.000000] Centaur CentaurHauls [ 0.000000] zhaoxin Shanghai [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bbff] usable [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078ec1fff] usable [ 0.000000] BIOS-e820: [mem 0x0000000078f74000-0x00000000798adfff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000798ae000-0x0000000079d4afff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000079d4b000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0 present. [ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0 12/17/2015 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 3499.750 MHz processor [ 0.000412] last_pfn = 0x880000 max_arch_pfn = 0x400000000 [ 0.000592] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.001880] last_pfn = 0x78ec2 max_arch_pfn = 0x400000000 [ 0.008227] found SMP MP-table at [mem 0x000fda20-0x000fda2f] [ 0.008240] Using GB pages for direct mapping [ 0.008324] RAMDISK: [mem 0x300cf000-0x3405efff] [ 0.008330] ACPI: Early table checksum verification disabled [ 0.008333] ACPI: RSDP 0x00000000000F0580 000024 (v02 SUPERM) [ 0.008337] ACPI: XSDT 0x00000000798AE098 0000B4 (v01 01072009 AMI 00010013) [ 0.008342] ACPI: FACP 0x00000000798E15B0 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008347] ACPI: DSDT 0x00000000798AE1E8 0333C4 (v02 SUPERM SMCI--MB 01072009 INTL 20091013) [ 0.008350] ACPI: FACS 0x0000000079D49F80 000040 [ 0.008353] ACPI: APIC 0x00000000798E16C0 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008355] ACPI: FPDT 0x00000000798E1788 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008358] ACPI: FIDT 0x00000000798E17D0 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.008361] ACPI: SPMI 0x00000000798E1870 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000) [ 0.008364] ACPI: MCFG 0x00000000798E18B0 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097) [ 0.008367] ACPI: UEFI 0x00000000798E18F0 000042 (v01 SUPERM SMCI--MB 01072009 00000000) [ 0.008369] ACPI: HPET 0x00000000798E1938 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.008372] ACPI: WDDT 0x00000000798E1970 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013) [ 0.008375] ACPI: SSDT 0x00000000798E19B0 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913) [ 0.008378] ACPI: SSDT 0x00000000798F8B30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913) [ 0.008381] ACPI: SSDT 0x00000000798FB180 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913) [ 0.008384] ACPI: PRAD 0x00000000798FB1E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913) [ 0.008386] ACPI: DMAR 0x00000000798FB2F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.008389] ACPI: HEST 0x00000000798FB3B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008392] ACPI: BERT 0x00000000798FB630 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008395] ACPI: ERST 0x00000000798FB660 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008398] ACPI: EINJ 0x00000000798FB890 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.008400] ACPI: Reserving FACP table memory at [mem 0x798e15b0-0x798e16bb] [ 0.008402] ACPI: Reserving DSDT table memory at [mem 0x798ae1e8-0x798e15ab] [ 0.008403] ACPI: Reserving FACS table memory at [mem 0x79d49f80-0x79d49fbf] [ 0.008404] ACPI: Reserving APIC table memory at [mem 0x798e16c0-0x798e1787] [ 0.008405] ACPI: Reserving FPDT table memory at [mem 0x798e1788-0x798e17cb] [ 0.008406] ACPI: Reserving FIDT table memory at [mem 0x798e17d0-0x798e186b] [ 0.008407] ACPI: Reserving SPMI table memory at [mem 0x798e1870-0x798e18af] [ 0.008408] ACPI: Reserving MCFG table memory at [mem 0x798e18b0-0x798e18eb] [ 0.008409] ACPI: Reserving UEFI table memory at [mem 0x798e18f0-0x798e1931] [ 0.008410] ACPI: Reserving HPET table memory at [mem 0x798e1938-0x798e196f] [ 0.008411] ACPI: Reserving WDDT table memory at [mem 0x798e1970-0x798e19af] [ 0.008412] ACPI: Reserving SSDT table memory at [mem 0x798e19b0-0x798f8b2e] [ 0.008413] ACPI: Reserving SSDT table memory at [mem 0x798f8b30-0x798fb17b] [ 0.008414] ACPI: Reserving SSDT table memory at [mem 0x798fb180-0x798fb1e3] [ 0.008415] ACPI: Reserving PRAD table memory at [mem 0x798fb1e8-0x798fb2e9] [ 0.008416] ACPI: Reserving DMAR table memory at [mem 0x798fb2f0-0x798fb3ab] [ 0.008417] ACPI: Reserving HEST table memory at [mem 0x798fb3b0-0x798fb62b] [ 0.008418] ACPI: Reserving BERT table memory at [mem 0x798fb630-0x798fb65f] [ 0.008419] ACPI: Reserving ERST table memory at [mem 0x798fb660-0x798fb88f] [ 0.008420] ACPI: Reserving EINJ table memory at [mem 0x798fb890-0x798fb9bf] [ 0.008514] No NUMA configuration found [ 0.008515] Faking a node at [mem 0x0000000000000000-0x000000087fffffff] [ 0.008524] NODE_DATA(0) allocated [mem 0x87ffd6000-0x87fffffff] [ 0.008855] Zone ranges: [ 0.008856] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.008858] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.008860] Normal [mem 0x0000000100000000-0x000000087fffffff] [ 0.008861] Device empty [ 0.008863] Movable zone start for each node [ 0.008865] Early memory node ranges [ 0.008866] node 0: [mem 0x0000000000001000-0x000000000008afff] [ 0.008868] node 0: [mem 0x0000000000100000-0x0000000078ec1fff] [ 0.008869] node 0: [mem 0x0000000100000000-0x000000087fffffff] [ 0.008873] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff] [ 0.008878] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.008913] On node 0, zone DMA: 117 pages in unavailable ranges [ 0.077944] On node 0, zone Normal: 28990 pages in unavailable ranges [ 0.078059] ACPI: PM-Timer IO Port: 0x408 [ 0.078066] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) [ 0.078068] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) [ 0.078069] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) [ 0.078070] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1]) [ 0.078071] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) [ 0.078071] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) [ 0.078073] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1]) [ 0.078073] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1]) [ 0.078083] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 [ 0.078087] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 [ 0.078089] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.078091] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.078095] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.078096] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 0.078100] TSC deadline timer available [ 0.078101] smpboot: Allowing 8 CPUs, 0 hotplug CPUs [ 0.078111] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.078113] PM: hibernation: Registered nosave memory: [mem 0x0008b000-0x000dffff] [ 0.078114] PM: hibernation: Registered nosave memory: [mem 0x000e0000-0x000fffff] [ 0.078115] PM: hibernation: Registered nosave memory: [mem 0x78ec2000-0x78f73fff] [ 0.078116] PM: hibernation: Registered nosave memory: [mem 0x78f74000-0x798adfff] [ 0.078117] PM: hibernation: Registered nosave memory: [mem 0x798ae000-0x79d4afff] [ 0.078118] PM: hibernation: Registered nosave memory: [mem 0x79d4b000-0x8fffffff] [ 0.078119] PM: hibernation: Registered nosave memory: [mem 0x90000000-0xfed1bfff] [ 0.078120] PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0xfed44fff] [ 0.078121] PM: hibernation: Registered nosave memory: [mem 0xfed45000-0xfeffffff] [ 0.078121] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xffffffff] [ 0.078123] [mem 0x90000000-0xfed1bfff] available for PCI devices [ 0.078125] Booting paravirtualized kernel on bare hardware [ 0.078127] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns [ 0.078133] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 [ 0.078344] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u262144 [ 0.078375] Built 1 zonelists, mobility grouping on. Total pages: 8228742 [ 0.078377] Policy zone: Normal [ 0.078378] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic root=UUID=7e9e2103-8bb7-4452-b35a-3b1230743254 ro pci=realloc=off console=tty0 console=ttyS1,115200 [ 0.078437] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-102-generic", will be passed to user space. [ 0.080585] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear) [ 0.081728] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear) [ 0.081815] mem auto-init: stack:off, heap alloc:on, heap free:off [ 0.166209] Memory: 32672704K/33438000K available (16393K kernel code, 4395K rwdata, 10948K rodata, 3352K init, 18716K bss, 765036K reserved, 0K cma-reserved) [ 0.166493] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1 [ 0.166508] Kernel/User page tables isolation: enabled [ 0.166539] ftrace: allocating 50694 entries in 199 pages [ 0.186124] ftrace: allocated 199 pages with 5 groups [ 0.186239] rcu: Hierarchical RCU implementation. [ 0.186241] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8. [ 0.186242] Rude variant of Tasks RCU enabled. [ 0.186243] Tracing variant of Tasks RCU enabled. [ 0.186244] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.186245] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 [ 0.189655] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16 [ 0.189853] random: crng init done [ 0.194731] Console: colour VGA+ 80x25 [ 0.205968] printk: console [tty0] enabled [ 1.177251] printk: console [ttyS1] enabled [ 1.181511] ACPI: Core revision 20210730 [ 1.185669] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns [ 1.194894] APIC: Switch to symmetric I/O mode setup [ 1.199939] DMAR: Host address width 46 [ 1.203851] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1 [ 1.209236] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de [ 1.217329] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff [ 1.223667] DMAR: ATSR flags: 0x0 [ 1.228142] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0 [ 1.234476] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0 [ 1.240898] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0 [ 1.247323] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000 [ 1.252790] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [ 1.252790] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [ 1.267971] DMAR-IR: Enabled IRQ remapping in xapic mode [ 1.273351] x2apic: IRQ remapping doesn't support X2APIC mode [ 1.279648] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 1.302901] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x327264bdb86, max_idle_ns: 440795319403 ns [ 1.313515] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.50 BogoMIPS (lpj=13999000) [ 1.317530] CPU0: Thermal monitoring enabled (TM1) [ 1.321566] process: using mwait in idle threads [ 1.325515] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 [ 1.329513] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 [ 1.333515] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 1.337514] Spectre V2 : Mitigation: Retpolines [ 1.341513] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 1.345513] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 1.349513] Spectre V2 : Enabling Restricted Speculation for firmware calls [ 1.353514] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier [ 1.357513] Spectre V2 : User space: Mitigation: STIBP via seccomp and prctl [ 1.361513] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp [ 1.365517] MDS: Mitigation: Clear CPU buffers [ 1.369513] TAA: Mitigation: Clear CPU buffers [ 1.373513] MMIO Stale Data: Mitigation: Clear CPU buffers [ 1.377518] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 1.381513] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 1.385513] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 1.389513] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 1.393513] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 1.415061] Freeing SMP alternatives memory: 44K [ 1.417514] pid_max: default: 32768 minimum: 301 [ 1.421534] LSM: Security Framework initializing [ 1.425521] landlock: Up and running. [ 1.429513] Yama: becoming mindful. [ 1.433530] AppArmor: AppArmor initialized [ 1.437573] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.441553] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.445870] smpboot: Estimated ratio of average max frequency by base frequency (times 1024): 1053 [ 1.449518] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1) [ 1.453621] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver. [ 1.457514] ... version: 3 [ 1.461513] ... bit width: 48 [ 1.465513] ... generic registers: 4 [ 1.469513] ... value mask: 0000ffffffffffff [ 1.473513] ... max period: 00007fffffffffff [ 1.477513] ... fixed-purpose events: 3 [ 1.481513] ... event mask: 000000070000000f [ 1.485573] signal: max sigframe size: 1776 [ 1.489528] rcu: Hierarchical SRCU implementation. [ 1.494230] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 1.497577] smp: Bringing up secondary CPUs ... [ 1.501594] x86: Booting SMP configuration: [ 1.505515] .... node #0, CPUs: #1 #2 #3 #4 [ 1.513594] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details. [ 1.521513] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details. [ 1.525513] MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details. [ 1.529610] #5 #6 #7 [ 1.530064] smp: Brought up 1 node, 8 CPUs [ 1.537515] smpboot: Max logical packages: 1 [ 1.541513] smpboot: Total of 8 processors activated (55996.00 BogoMIPS) [ 1.549633] devtmpfs: initialized [ 1.552925] x86/mm: Memory block size: 128MB [ 1.555112] ACPI: PM: Registering ACPI NVS region [mem 0x798ae000-0x79d4afff] (4837376 bytes) [ 1.557589] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 1.561524] futex hash table entries: 2048 (order: 5, 131072 bytes, linear) [ 1.565559] pinctrl core: initialized pinctrl subsystem [ 1.569603] PM: RTC time: 22:25:38, date: 2024-04-18 [ 1.573619] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 1.577845] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations [ 1.581811] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 1.585844] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 1.589524] audit: initializing netlink subsys (disabled) [ 1.593531] audit: type=2000 audit(1713479137.356:1): state=initialized audit_enabled=0 res=1 [ 1.593628] thermal_sys: Registered thermal governor 'fair_share' [ 1.597516] thermal_sys: Registered thermal governor 'bang_bang' [ 1.601514] thermal_sys: Registered thermal governor 'step_wise' [ 1.605513] thermal_sys: Registered thermal governor 'user_space' [ 1.609513] thermal_sys: Registered thermal governor 'power_allocator' [ 1.613518] EISA bus registered [ 1.620722] cpuidle: using governor ladder [ 1.621515] cpuidle: using governor menu [ 1.625529] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 1.629514] ACPI: bus type PCI registered [ 1.633515] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 1.637568] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000) [ 1.641515] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820 [ 1.645527] PCI: Using configuration type 1 for base access [ 1.650537] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 1.653530] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 1.657514] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 1.661588] ACPI: Added _OSI(Module Device) [ 1.665515] ACPI: Added _OSI(Processor Device) [ 1.669521] ACPI: Added _OSI(3.0 _SCP Extensions) [ 1.673513] ACPI: Added _OSI(Processor Aggregator Device) [ 1.677513] ACPI: Added _OSI(Linux-Dell-Video) [ 1.681513] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 1.685513] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 1.736545] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 1.740837] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored [ 1.745664] ACPI: Dynamic OEM Table Load: [ 1.777503] ACPI: Interpreter enabled [ 1.777524] ACPI: PM: (supports S0 S4 S5) [ 1.781514] ACPI: Using IOAPIC for interrupt routing [ 1.785536] HEST: Enabling Firmware First mode for corrected errors. [ 1.789592] HEST: Table parsing has been initialized. [ 1.793661] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. [ 1.797515] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 1.801513] PCI: Using E820 reservations for host bridge windows [ 1.806200] ACPI: Enabled 5 GPEs in block 00 to 3F [ 1.842831] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff]) [ 1.845516] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 1.851496] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 1.854566] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.857514] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.861544] PCI host bridge to bus 0000:ff [ 1.865514] pci_bus 0000:ff: root bus resource [bus ff] [ 1.869524] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000 [ 1.873579] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100 [ 1.877566] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100 [ 1.881565] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000 [ 1.885566] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000 [ 1.889568] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000 [ 1.893564] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000 [ 1.897565] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000 [ 1.901568] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000 [ 1.905564] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000 [ 1.909565] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000 [ 1.913564] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000 [ 1.917565] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000 [ 1.921566] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000 [ 1.925566] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100 [ 1.929566] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000 [ 1.933567] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100 [ 1.937564] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000 [ 1.941564] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000 [ 1.945539] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100 [ 1.949575] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000 [ 1.953615] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000 [ 1.957578] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000 [ 1.961577] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000 [ 1.965580] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000 [ 1.969577] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000 [ 1.973577] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000 [ 1.977661] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000 [ 1.981571] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000 [ 1.985580] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000 [ 1.989578] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000 [ 1.993579] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000 [ 1.997579] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000 [ 2.001566] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000 [ 2.005566] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000 [ 2.009566] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000 [ 2.013569] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000 [ 2.017579] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000 [ 2.021577] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000 [ 2.025579] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000 [ 2.029581] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000 [ 2.033571] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000 [ 2.037566] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000 [ 2.041568] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000 [ 2.045569] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000 [ 2.049567] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000 [ 2.053567] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000 [ 2.057567] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000 [ 2.061572] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000 [ 2.065568] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000 [ 2.069565] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000 [ 2.073566] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000 [ 2.077539] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000 [ 2.081567] pci 0000:ff:1f.0: [8086:6f88] type 00 class 0x088000 [ 2.085566] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000 [ 2.093968] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe]) [ 2.097516] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 2.101854] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 2.105986] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 2.109514] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration [ 2.113803] PCI host bridge to bus 0000:00 [ 2.117515] pci_bus 0000:00: root bus resource [bus 00-fe] [ 2.121513] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 2.125513] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window] [ 2.129513] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 2.133513] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window] [ 2.137513] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window] [ 2.141513] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window] [ 2.145620] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000 [ 2.149724] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400 [ 2.153579] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold [ 2.157689] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400 [ 2.161578] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold [ 2.165679] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400 [ 2.169578] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold [ 2.173674] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000 [ 2.177532] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit] [ 2.181708] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000 [ 2.185532] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit] [ 2.189705] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000 [ 2.193532] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit] [ 2.197706] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000 [ 2.201532] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit] [ 2.205704] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000 [ 2.209532] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit] [ 2.213710] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000 [ 2.217532] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit] [ 2.221705] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000 [ 2.225532] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit] [ 2.229705] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000 [ 2.233533] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit] [ 2.237658] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000 [ 2.241735] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000 [ 2.245661] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000 [ 2.249683] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020 [ 2.253528] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff] [ 2.257754] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000 [ 2.261806] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601 [ 2.265535] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117] [ 2.269525] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103] [ 2.273525] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7] [ 2.277525] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3] [ 2.281525] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f] [ 2.285525] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff] [ 2.289553] pci 0000:00:11.4: PME# supported from D3hot [ 2.293669] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330 [ 2.297540] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit] [ 2.301589] pci 0000:00:14.0: PME# supported from D3hot D3cold [ 2.305662] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000 [ 2.309539] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit] [ 2.313590] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold [ 2.317658] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000 [ 2.321539] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit] [ 2.325590] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold [ 2.329671] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320 [ 2.333536] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff] [ 2.337618] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold [ 2.341674] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400 [ 2.345603] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold [ 2.349676] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400 [ 2.353606] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold [ 2.357675] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400 [ 2.361605] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold [ 2.365684] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320 [ 2.369536] pci 0000:00:1d.0: reg 0x10: [mem 0xfb433000-0xfb4333ff] [ 2.373618] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold [ 2.377670] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100 [ 2.381789] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601 [ 2.385533] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077] [ 2.389524] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063] [ 2.393523] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057] [ 2.397523] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043] [ 2.401523] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f] [ 2.405524] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff] [ 2.409547] pci 0000:00:1f.2: PME# supported from D3hot [ 2.413657] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500 [ 2.417539] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit] [ 2.421545] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f] [ 2.425728] acpiphp: Slot [1] registered [ 2.429530] pci 0000:00:01.0: PCI bridge to [bus 01] [ 2.433692] pci 0000:02:00.0: [8086:0953] type 00 class 0x010802 [ 2.437534] pci 0000:02:00.0: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit] [ 2.441555] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref] [ 2.445516] pci 0000:02:00.0: enabling Extended Tags [ 2.449618] pci 0000:00:02.0: PCI bridge to [bus 02] [ 2.453515] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 2.457695] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000 [ 2.461535] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref] [ 2.465522] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f] [ 2.469536] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref] [ 2.473523] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref] [ 2.477554] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold [ 2.481534] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.485513] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 2.489526] pci 0000:03:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.493513] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs) [ 2.497676] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000 [ 2.501535] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref] [ 2.505522] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f] [ 2.509536] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref] [ 2.513522] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref] [ 2.517554] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold [ 2.521530] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.525513] pci 0000:03:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 2.529526] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.533514] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs) [ 2.537646] pci 0000:00:03.0: PCI bridge to [bus 03] [ 2.541514] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 2.545514] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 2.549515] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 2.553544] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 2.557579] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400 [ 2.561602] pci 0000:05:00.0: enabling Extended Tags [ 2.565586] pci 0000:05:00.0: supports D1 D2 [ 2.569513] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 2.573604] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 2.577515] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 2.581515] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 2.585561] pci_bus 0000:06: extended config space not accessible [ 2.589543] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000 [ 2.593547] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff] [ 2.597531] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff] [ 2.601530] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f] [ 2.605690] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 2.609556] pci 0000:06:00.0: supports D1 D2 [ 2.613513] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 2.617628] pci 0000:05:00.0: PCI bridge to [bus 06] [ 2.621519] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 2.625516] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 2.629612] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000 [ 2.633540] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff] [ 2.637541] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f] [ 2.641527] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff] [ 2.645649] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold [ 2.649552] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.653514] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 2.657539] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.661513] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 2.665690] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000 [ 2.669540] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff] [ 2.673541] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f] [ 2.677528] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff] [ 2.681644] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold [ 2.685551] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 2.689513] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 2.693539] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 2.697513] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 2.701670] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 2.705515] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 2.709515] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 2.713517] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08 [ 2.718436] ACPI: PCI: Interrupt link LNKA configured for IRQ 11 [ 2.721563] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 2.725562] ACPI: PCI: Interrupt link LNKC configured for IRQ 5 [ 2.729560] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 2.733563] ACPI: PCI: Interrupt link LNKE configured for IRQ 0 [ 2.737513] ACPI: PCI: Interrupt link LNKE disabled [ 2.741561] ACPI: PCI: Interrupt link LNKF configured for IRQ 0 [ 2.745513] ACPI: PCI: Interrupt link LNKF disabled [ 2.749561] ACPI: PCI: Interrupt link LNKG configured for IRQ 0 [ 2.753513] ACPI: PCI: Interrupt link LNKG disabled [ 2.757561] ACPI: PCI: Interrupt link LNKH configured for IRQ 0 [ 2.761513] ACPI: PCI: Interrupt link LNKH disabled [ 2.765701] iommu: Default domain type: Translated [ 2.769514] iommu: DMA domain TLB invalidation policy: lazy mode [ 2.773641] SCSI subsystem initialized [ 2.777522] pci 0000:06:00.0: vgaarb: setting as boot VGA device [ 2.781513] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none [ 2.781514] pci 0000:06:00.0: vgaarb: bridge control possible [ 2.785513] vgaarb: loaded [ 2.788301] ACPI: bus type USB registered [ 2.789523] usbcore: registered new interface driver usbfs [ 2.793517] usbcore: registered new interface driver hub [ 2.797518] usbcore: registered new device driver usb [ 2.801528] pps_core: LinuxPPS API ver. 1 registered [ 2.805513] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 2.809516] PTP clock support registered [ 2.813527] EDAC MC: Ver: 3.0.0 [ 2.816827] NetLabel: Initializing [ 2.817514] NetLabel: domain hash size = 128 [ 2.821513] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 2.825524] NetLabel: unlabeled traffic allowed by default [ 2.829522] PCI: Using ACPI for IRQ routing [ 2.838803] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0 [ 2.841514] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [ 2.847552] clocksource: Switched to clocksource tsc-early [ 2.859680] VFS: Disk quotas dquot_6.6.0 [ 2.863682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.870725] AppArmor: AppArmor Filesystem Enabled [ 2.875523] pnp: PnP ACPI init [ 2.879106] system 00:01: [io 0x0500-0x057f] has been reserved [ 2.885094] system 00:01: [io 0x0400-0x047f] has been reserved [ 2.891082] system 00:01: [io 0x0580-0x059f] has been reserved [ 2.897072] system 00:01: [io 0x0600-0x061f] has been reserved [ 2.903058] system 00:01: [io 0x0880-0x0883] has been reserved [ 2.909048] system 00:01: [io 0x0800-0x081f] has been reserved [ 2.915037] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved [ 2.921722] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved [ 2.928404] system 00:01: [mem 0xff000000-0xffffffff] has been reserved [ 2.935086] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved [ 2.941769] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved [ 2.948451] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved [ 2.955135] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved [ 2.962028] system 00:02: [io 0x0a00-0x0a0f] has been reserved [ 2.968014] system 00:02: [io 0x0a10-0x0a1f] has been reserved [ 2.974001] system 00:02: [io 0x0a20-0x0a2f] has been reserved [ 2.979992] system 00:02: [io 0x0a30-0x0a3f] has been reserved [ 2.985982] system 00:02: [io 0x0a40-0x0a4f] has been reserved [ 2.992949] pnp: PnP ACPI: found 5 devices [ 3.002500] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 3.011503] NET: Registered PF_INET protocol family [ 3.016611] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 3.025904] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, linear) [ 3.034728] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 3.042724] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 3.051130] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear) [ 3.058663] TCP: Hash tables configured (established 262144 bind 65536) [ 3.065475] MPTCP token hash table entries: 32768 (order: 7, 786432 bytes, linear) [ 3.073252] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 3.080270] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 3.087727] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 3.093462] NET: Registered PF_XDP protocol family [ 3.098340] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000 [ 3.109915] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref] [ 3.117828] pci 0000:00:01.0: PCI bridge to [bus 01] [ 3.122872] pci 0000:00:02.0: PCI bridge to [bus 02] [ 3.127906] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 3.134772] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.142424] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.150426] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.158164] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.166254] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.173906] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.181907] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.189645] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.197735] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.205384] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.213386] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.221124] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.229213] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 3.236865] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 3.244868] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 3.252606] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 3.260695] pci 0000:00:03.0: PCI bridge to [bus 03] [ 3.265722] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 3.271883] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 3.278740] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 3.286573] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 3.291622] pci 0000:05:00.0: PCI bridge to [bus 06] [ 3.296649] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 3.302812] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 3.309686] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 3.314980] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 3.321142] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 3.328006] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref] [ 3.335835] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref] [ 3.343758] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref] [ 3.351593] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref] [ 3.359515] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 3.364549] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 3.370710] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 3.377566] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref] [ 3.385402] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 3.391646] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window] [ 3.397897] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 3.412523] pci_bus 0000:00: resource 7 [mem 0x90000000-0xfbffbfff window] [ 3.419461] pci_bus 0000:00: resource 8 [mem 0xfedb0000-0xfedb000f window] [ 3.426404] pci_bus 0000:00: resource 9 [mem 0xfedc0000-0xfedc000f window] [ 3.433347] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff] [ 3.439681] pci_bus 0000:03: resource 0 [io 0xe000-0xefff] [ 3.445322] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff] [ 3.451651] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref] [ 3.458961] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff] [ 3.464599] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff] [ 3.470935] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff] [ 3.476580] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff] [ 3.482915] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff] [ 3.488555] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff] [ 3.494884] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref] [ 3.502508] pci 0000:00:05.0: disabled boot interrupts on device [8086:6f28] [ 3.531824] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x160 took 21521 usecs [ 3.563809] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x160 took 23914 usecs [ 3.571310] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes [ 3.577861] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.577870] Trying to unpack rootfs image as initramfs... [ 3.584368] software IO TLB: mapped [mem 0x0000000074ec2000-0x0000000078ec2000] (64MB) [ 3.598308] Initialise system trusted keyrings [ 3.602830] Key type blacklist registered [ 3.606950] workingset: timestamp_bits=36 max_order=23 bucket_order=0 [ 3.614419] zbud: loaded [ 3.617228] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 3.623289] fuse: init (API version 7.34) [ 3.627512] integrity: Platform Keyring initialized [ 3.640148] Key type asymmetric registered [ 3.644316] Asymmetric key parser 'x509' registered [ 3.649284] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243) [ 3.656809] io scheduler mq-deadline registered [ 3.661877] pcieport 0000:00:01.0: PME: Signaling with IRQ 25 [ 3.667912] pcieport 0000:00:02.0: PME: Signaling with IRQ 27 [ 3.673935] pcieport 0000:00:03.0: PME: Signaling with IRQ 29 [ 3.679910] pcieport 0000:00:1c.0: PME: Signaling with IRQ 30 [ 3.685889] pcieport 0000:00:1c.2: PME: Signaling with IRQ 31 [ 3.691850] pcieport 0000:00:1c.4: PME: Signaling with IRQ 32 [ 3.697813] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.704716] ACPI: \_SB_.SCK0.CP00: Found 2 idle states [ 3.710461] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0 [ 3.718936] ACPI: button: Power Button [PWRB] [ 3.723395] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 [ 3.730911] ACPI: button: Power Button [PWRF] [ 3.736494] ERST: Error Record Serialization Table (ERST) support is initialized. [ 3.744068] pstore: Registered erst as persistent store backend [ 3.750184] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled [ 3.777121] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.806029] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.815613] Linux agpgart interface v0.103 [ 3.821539] loop: module loaded [ 3.824974] tun: Universal TUN/TAP device driver, 1.6 [ 3.830116] PPP generic driver version 2.4.2 [ 3.834497] VFIO - User Level meta-driver version: 0.3 [ 3.839793] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 3.846393] ehci-pci: EHCI PCI platform driver [ 3.851056] ehci-pci 0000:00:1a.0: EHCI Host Controller [ 3.856359] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1 [ 3.863856] ehci-pci 0000:00:1a.0: debug port 2 [ 3.872385] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000 [ 3.891703] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00 [ 3.897560] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15 [ 3.905915] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.913230] usb usb1: Product: EHCI Host Controller [ 3.918179] usb usb1: Manufacturer: Linux 5.15.0-102-generic ehci_hcd [ 3.924693] usb usb1: SerialNumber: 0000:00:1a.0 [ 3.929521] hub 1-0:1.0: USB hub found [ 3.931689] Freeing initrd memory: 65088K [ 3.933347] hub 1-0:1.0: 2 ports detected [ 3.941752] ehci-pci 0000:00:1d.0: EHCI Host Controller [ 3.947046] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2 [ 3.954538] ehci-pci 0000:00:1d.0: debug port 2 [ 3.963020] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000 [ 3.983738] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00 [ 3.989590] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15 [ 3.997945] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 4.005248] usb usb2: Product: EHCI Host Controller [ 4.010190] usb usb2: Manufacturer: Linux 5.15.0-102-generic ehci_hcd [ 4.016699] usb usb2: SerialNumber: 0000:00:1d.0 [ 4.021490] hub 2-0:1.0: USB hub found [ 4.025314] hub 2-0:1.0: 2 ports detected [ 4.029505] ehci-platform: EHCI generic platform driver [ 4.034816] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 4.041062] ohci-pci: OHCI PCI platform driver [ 4.045591] ohci-platform: OHCI generic platform driver [ 4.050882] uhci_hcd: USB Universal Host Controller Interface driver [ 4.057350] i8042: PNP: No PS/2 controller found. [ 4.062185] mousedev: PS/2 mouse device common for all mice [ 4.067898] rtc_cmos 00:00: RTC can wake from S4 [ 4.072803] rtc_cmos 00:00: registered as rtc0 [ 4.077342] rtc_cmos 00:00: setting system clock to 2024-04-18T22:25:42 UTC (1713479142) [ 4.085545] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram [ 4.092318] i2c_dev: i2c /dev entries driver [ 4.096828] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. [ 4.109109] device-mapper: uevent: version 1.0.3 [ 4.113858] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com [ 4.122402] platform eisa.0: Probing EISA bus 0 [ 4.127003] platform eisa.0: EISA: Cannot allocate resource for mainboard [ 4.133855] platform eisa.0: Cannot allocate resource for EISA slot 1 [ 4.140363] platform eisa.0: Cannot allocate resource for EISA slot 2 [ 4.146874] platform eisa.0: Cannot allocate resource for EISA slot 3 [ 4.153382] platform eisa.0: Cannot allocate resource for EISA slot 4 [ 4.159891] platform eisa.0: Cannot allocate resource for EISA slot 5 [ 4.166402] platform eisa.0: Cannot allocate resource for EISA slot 6 [ 4.172910] platform eisa.0: Cannot allocate resource for EISA slot 7 [ 4.179420] platform eisa.0: Cannot allocate resource for EISA slot 8 [ 4.185929] platform eisa.0: EISA: Detected 0 cards [ 4.190880] intel_pstate: Intel P-state driver initializing [ 4.197460] ledtrig-cpu: registered to indicate activity on CPUs [ 4.203681] drop_monitor: Initializing network drop monitor service [ 4.210126] NET: Registered PF_INET6 protocol family [ 4.222449] Segment Routing with IPv6 [ 4.226210] In-situ OAM (IOAM) with IPv6 [ 4.230212] NET: Registered PF_PACKET protocol family [ 4.235434] Key type dns_resolver registered [ 4.240190] microcode: sig=0x406f1, pf=0x1, revision=0xb000040 [ 4.246164] microcode: Microcode Update Driver: v2.2. [ 4.246242] resctrl: L3 allocation detected [ 4.255600] resctrl: L3 monitoring detected [ 4.259852] IPI shorthand broadcast: enabled [ 4.264207] sched_clock: Marking stable (3216524796, 1047665431)->(5253335923, -989145696) [ 4.272784] registered taskstats version 1 [ 4.275708] usb 1-1: new high-speed USB device number 2 using ehci-pci [ 4.283668] Loading compiled-in X.509 certificates [ 4.289099] Loaded X.509 cert 'Build time autogenerated kernel key: 8dbd8aa72291a99043f1138db79927b696791b59' [ 4.299531] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969' [ 4.309765] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19' [ 4.319852] blacklist: Loading compiled-in revocation X.509 certificates [ 4.326632] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0' [ 4.336552] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03' [ 4.347087] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b' [ 4.357961] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8' [ 4.367726] usb 2-1: new high-speed USB device number 2 using ehci-pci [ 4.368495] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d' [ 4.385878] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c' [ 4.396675] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af' [ 4.407469] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9' [ 4.419292] zswap: loaded using pool lzo/zbud [ 4.424023] Key type .fscrypt registered [ 4.428032] Key type fscrypt-provisioning registered [ 4.433176] pstore: Using crash dump compression: deflate [ 4.441752] Key type encrypted registered [ 4.445833] AppArmor: AppArmor sha1 policy hashing enabled [ 4.451397] ima: No TPM chip found, activating TPM-bypass! [ 4.456954] Loading compiled-in module X.509 certificates [ 4.457021] usb 1-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05 [ 4.462877] Loaded X.509 cert 'Build time autogenerated kernel key: 8dbd8aa72291a99043f1138db79927b696791b59' [ 4.470697] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 4.488479] ima: Allocated hash algorithm: sha1 [ 4.500295] ima: No architecture policies found [ 4.500373] hub 1-1:1.0: USB hub found [ 4.504912] evm: Initialising EVM extended attributes: [ 4.513942] evm: security.selinux [ 4.517330] evm: security.SMACK64 [ 4.520719] evm: security.SMACK64EXEC [ 4.524455] evm: security.SMACK64TRANSMUTE [ 4.524501] hub 1-1:1.0: 6 ports detected [ 4.528617] evm: security.SMACK64MMAP [ 4.528617] evm: security.apparmor [ 4.528619] evm: security.ima [ 4.528619] evm: security.capability [ 4.528620] evm: HMAC attrs: 0x1 [ 4.528881] PM: Magic number: 4:51:452 [ 4.554291] RAS: Correctable Errors collector initialized. [ 4.560076] usb 2-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05 [ 4.568350] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 4.575835] Freeing unused decrypted memory: 2036K [ 4.575847] hub 2-1:1.0: USB hub found [ 4.584576] hub 2-1:1.0: 8 ports detected [ 4.584906] Freeing unused kernel image (initmem) memory: 3352K [ 4.599686] Write protecting the kernel read-only data: 30720k [ 4.605971] Freeing unused kernel image (text/rodata gap) memory: 2036K [ 4.612887] Freeing unused kernel image (rodata/data gap) memory: 1340K [ 4.627720] tsc: Refined TSC clocksource calibration: 3499.996 MHz [ 4.633997] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734d20948, max_idle_ns: 440795306379 ns [ 4.662881] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 4.669392] x86/mm: Checking user space page tables [ 4.674348] clocksource: Switched to clocksource tsc [ 4.711609] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 4.718119] Run /init as init process Loading, please wait... Starting version 249.11-0ubuntu3.6 [ 4.885819] dca service started, version 1.12.1 [ 4.891909] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 4.898318] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3 [ 4.906889] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810 [ 4.907040] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode [ 4.916603] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 4.924300] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst [ 4.937414] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4 [ 4.937490] igb: Intel(R) Gigabit Ethernet Network Driver [ 4.944910] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed [ 4.950371] igb: Copyright (c) 2007-2014 Intel Corporation. [ 4.962384] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15 [ 4.963967] scsi host0: ahci [ 4.970746] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 4.973812] scsi host1: ahci [ 4.981004] usb usb3: Product: xHCI Host Controller [ 4.984054] scsi host2: ahci [ 4.988898] usb usb3: Manufacturer: Linux 5.15.0-102-generic xhci-hcd [ 4.991921] scsi host3: ahci [ 4.998354] usb usb3: SerialNumber: 0000:00:14.0 [ 5.001344] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 33 [ 5.004236] igb 0000:07:00.0: added PHC on eth0 [ 5.004260] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection [ 5.004261] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:bc [ 5.004338] igb 0000:07:00.0: eth0: PBA No: 050B00-000 [ 5.004339] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 5.006144] hub 3-0:1.0: USB hub found [ 5.013494] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 33 [ 5.018132] hub 3-0:1.0: 15 ports detected [ 5.025041] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 33 [ 5.034628] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.15 [ 5.037447] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 33 [ 5.045106] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 5.049066] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver [ 5.049181] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode [ 5.049184] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst [ 5.056416] usb usb4: Product: xHCI Host Controller [ 5.056418] usb usb4: Manufacturer: Linux 5.15.0-102-generic xhci-hcd [ 5.056420] usb usb4: SerialNumber: 0000:00:14.0 [ 5.058348] igb 0000:07:00.1: added PHC on eth1 [ 5.058375] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection [ 5.058377] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:d9:bd [ 5.058452] igb 0000:07:00.1: eth1: PBA No: 050B00-000 [ 5.058454] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 5.060584] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 5.068168] hub 4-0:1.0: USB hub found [ 5.112200] scsi host4: ahci [ 5.113312] hub 4-0:1.0: 6 ports detected [ 5.118338] scsi host5: ahci [ 5.141066] nvme nvme0: pci function 0000:02:00.0 [ 5.141124] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt [ 5.142101] i2c i2c-2: 2/8 memory slots populated (from DMI) [ 5.142103] i2c i2c-2: Systems with more than 4 memory slots not supported yet, not instantiating SPD [ 5.142126] igb 0000:07:00.0 eno1: renamed from eth0 [ 5.148303] scsi host6: ahci [ 5.156228] nvme nvme0: 8/0/0 default/read/poll queues [ 5.161169] scsi host7: ahci [ 5.222532] scsi host8: ahci [ 5.225603] scsi host9: ahci [ 5.228574] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 54 [ 5.236056] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 54 [ 5.24354ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 54 [ 5.251021] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 54 [ 5.258500] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 54 [ 5.265980] ata10: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432380 irq 54 [ 5.362994] ata4: SATA link down (SStatus 0 SControl 300) [ 5.368502] ata1: SATA link down (SStatus 0 SControl 300) [ 5.374000] ata2: SATA link down (SStatus 0 SControl 300) [ 5.375671] usb 3-12: new high-speed USB device number 2 using xhci_hcd [ 5.379500] ata3: SATA link down (SStatus 0 SControl 300) [ 5.531937] usb 3-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00 [ 5.540314] usb 3-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 5.548043] hub 3-12:1.0: USB hub found [ 5.552035] hub 3-12:1.0: 4 ports detected [ 5.586137] ata8: SATA link down (SStatus 0 SControl 300) [ 5.591643] ata9: SATA link down (SStatus 0 SControl 300) [ 5.597150] ata6: SATA link down (SStatus 0 SControl 300) [ 5.602643] ata7: SATA link down (SStatus 0 SControl 300) [ 5.608134] ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 5.614413] ata10: SATA link down (SStatus 0 SControl 300) [ 5.620160] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133 [ 5.626996] ata5.00: NCQ Send/Recv Log not supported [ 5.632056] ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 32), AA [ 5.640462] ata5.00: NCQ Send/Recv Log not supported [ 5.645518] ata5.00: configured for UDMA/133 [ 5.650093] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5 [ 5.658555] sd 4:0:0:0: Attached scsi generic sg0 type 0 [ 5.658733] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB) [ 5.671724] sd 4:0:0:0: [sda] Write Protect is off [ 5.676602] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 5.700408] sda: sda1 [ 5.703094] sd 4:0:0:0: [sda] Attached SCSI disk [ 5.709366] igb 0000:07:00.1 eno2: renamed from eth1 [ 5.879725] usb 3-12.1: new low-speed USB device number 3 using xhci_hcd [ 6.012173] usb 3-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00 [ 6.020722] usb 3-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 6.039518] hid: raw HID events driver (C) Jiri Kosina [ 6.047520] usbcore: registered new interface driver usbhid [ 6.053184] usbhid: USB HID core driver [ 6.059675] i HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.0/0003:0557:2419.0001/input/input2 [ 6.127897] hid-generic 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0 [ 6.139959] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-12/3-12.1/3-12.1:1.1/0003:0557:2419.0002/input/input3 [ 6.151886] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1 [ 6.264561] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 6.274962] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 6.283584] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF [ 6.289839] ixgbe 0000:03:00.0: 0c:c4:7a:8f:cc:0c [ 6.306223] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection [ 6.476499] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 6.486902] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 6.495526] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF [ 6.502656] ixgbe 0000:03:00.1: 0c:c4:7a:8f:cc:0d [ 6.511018] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection [ 6.519309] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1 [ 6.540281] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done. Begin: Will now check root file system ... fsck from util-linux 2.37.2 [/usr/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1 /dev/sda1: recovering journal /dev/sda1: Clearing orphaned inode 24691 (uid=0, gid=0, mode=0100644, size=4447536) /dev/sda1: clean, 198140/61054976 files, 6688498/244190336 blocks done. [ 13.680307] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none. done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... done. [ 14.505308] systemd[1]: Inserted module 'autofs4' [ 14.602885] systemd[1]: systemd 249.11-0ubuntu3.6 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 14.651797] systemd[1]: Detected architecture x86-64. Welcome to Ubuntu 22.04.1 LTS! [ 14.719401] systemd[1]: Hostname set to . [ 16.114444] systemd[1]: Queued start job for default target Graphical Interface. [ 16.124619] systemd[1]: Created slice Slice /system/modprobe. [ OK ] Created slice Slice /system/modprobe. [ 16.148624] systemd[1]: Created slice Slice /system/serial-getty. [ OK ] Created slice Slice /system/serial-getty. [ 16.172482] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ 16.195879] systemd[1]: Started ntp-systemd-netif.path. [ OK ] Started ntp-systemd-netif.path. [ 16.215925] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password R…uests to Wall Directory Watch. [ 16.240127] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. [ OK ] Set up automount Arbitrary…s File System Automount Point. [ 16.267865] systemd[1]: Reached target ceph target allowing to start/stop all ceph-fuse@.service instances at once. [ OK ] Reached target ceph target…se@.service instances at once. [ 16.295822] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mon@.service instances at once. [ OK ] Reached target ceph target…on@.service instances at once. [ 16.323818] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mds@.service instances at once. [ OK ] Reached target ceph target…ds@.service instances at once. [ 16.351840] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mgr@.service instances at once. [ OK ] Reached target ceph target…gr@.service instances at once. [ 16.379809] systemd[1]: Reached target ceph target allowing to start/stop all ceph-osd@.service instances at once. [ OK ] Reached target ceph target…sd@.service instances at once. [ 16.407805] systemd[1]: Reached target ceph target allowing to start/stop all ceph-radosgw@.service instances at once. [ OK ] Reached target ceph target…gw@.service instances at once. [ 16.435901] systemd[1]: Reached target User and Group Name Lookups. [ OK ] Reached target User and Group Name Lookups. [ 16.459839] systemd[1]: Reached target Slice Units. [ OK ] Reached target Slice Units. [ 16.479878] systemd[1]: Reached target Local Verity Protected Volumes. [ OK ] Reached target Local Verity Protected Volumes. [ 16.503940] systemd[1]: Listening on Device-mapper event daemon FIFOs. [ OK ] Listening on Device-mapper event daemon FIFOs. [ 16.528060] systemd[1]: Listening on LVM2 poll daemon socket. [ OK ] Listening on LVM2 poll daemon socket. [ 16.554526] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ 16.580079] systemd[1]: Listening on Syslog Socket. [ OK ] Listening on Syslog Socket. [ 16.600048] systemd[1]: Listening on fsck to fsckd communication Socket. [ OK ] Listening on fsck to fsckd communication Socket. [ 16.623964] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ 16.648241] systemd[1]: Listening on Journal Audit Socket. [ OK ] Listening on Journal Audit Socket. [ 16.668037] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 16.692076] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 16.712187] systemd[1]: Listening on Network Service Netlink Socket. [ OK ] Listening on Network Service Netlink Socket. [ 16.736099] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 16.756013] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 16.777513] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ 16.797552] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... [ 16.821262] systemd[1]: Mounting NFSD configuration filesystem... Mounting NFSD configuration filesystem... [ 16.845451] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ 16.865586] systemd[1]: Mounting Kernel Trace File System... Mounting Kernel Trace File System... [ 16.880738] RPC: Registered named UNIX socket transport module. [ 16.886737] RPC: Registered udp transport module. [ 16.891515] RPC: Registered tcp transport module. [ 16.896293] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 16.904181] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.923819] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped. [ 16.934202] systemd[1]: Starting Set the console keyboard layout... Starting Set the console keyboard layout... [ 16.956844] systemd[1]: Starting Create List of Static Device Nodes... Starting Create List of Static Device Nodes... [ 16.980436] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... Starting Monitoring of LVM…meventd or progress polling... [ 17.008848] systemd[1]: Starting Load Kernel Module chromeos_pstore... Starting Load Kernel Module chromeos_pstore... [ 17.032512] systemd[1]: Starting Load Kernel Module configfs... Starting Load Kernel Module configfs... [ 17.052484] systemd[1]: Starting Load Kernel Module drm... Starting Load Kernel Module drm... [ 17.072485] systemd[1]: Starting Load Kernel Module efi_pstore... Starting Load Kernel Module efi_pstore... [ 17.096877] systemd[1]: Starting Load Kernel Module fuse... [ 17.097730] Installing knfsd (copyright (C) 1996 okir@monad.swb.de). Starting Load Kernel Module fuse... [ 17.125285] systemd[1]: Starting Load Kernel Module pstore_blk... Starting Load Kernel Module pstore_blk... [ 17.149185] systemd[1]: Starting Load Kernel Module pstore_zone... Starting Load Kernel Module pstore_zone... [ 17.173153] systemd[1]: Starting Load Kernel Module ramoops... Starting Load Kernel Module ramoops... [ 17.191855] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped. [ 17.201709] systemd[1]: Condition check resulted in File System Check on Root Device being skipped. [ 17.212800] systemd[1]: Starting Load Kernel Modules... Starting Load Kernel Modules... [ 17.233096] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ 17.256614] systemd[1]: Starting Coldplug All udev Devices... Starting Coldplug All udev Devices... [ 17.277551] systemd[1]: Started Journal Service. [ OK ] Started Journal Service. [ 17.286907] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro,user_xattr. Quota mode: none. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ 17.323552] lp: driver loaded but no devices found [ OK ] Mounted NFSD configuration filesystem. [ OK ] Mounted Kernel Debug File System. [ 17.357248] Loading iSCSI transport class v2.0-870. [ OK ] Mounted Kernel Trace File System. [ OK ] Finished Set the console keyboard layout. [ OK ] Finished Create List of Static Device Nodes. [ OK ] Finished Monitoring of LVM… dmeventd or progress polling. [ OK ] Finished Load Kernel Module chromeos_pstore. [ OK ] Finished Load Kernel Module configfs. [ OK ] Finished Load Kernel Module drm. [ OK ] Finished Load Kernel Module efi_pstore. [ OK ] Finished Load Kernel Module fuse. [ OK ] Finished Load Kernel Module pstore_blk. [ OK ] Finished Load Kernel Module pstore_zone. [ OK ] Finished Load Kernel Module ramoops. [ OK ] Finished Load Kernel Modules. [ OK ] Finished Remount Root and Kernel File Systems. Activating swap /swapfile... Mounting FUSE Control File System... Mounting Kernel Configuration File System... Starting Initial Check File System Quotas... Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Apply Kernel Variables... [ 19.431026] Adding 2097148k swap on /swapfile. Priority:-2 extents:6 across:2260988k FS Starting Create System Users[[ 19.444842] systemd-journald[366]: Received client request to flush runtime journal. 0m... [ OK ] Activated swap /swapfile. [ OK ] Finished Coldplug All udev Devices. [ OK ] Mounted FUSE Control File System. [ OK ] Mounted Kernel Configuration File System. [ OK ] Reached target Swaps. Starting Helper to synchronize boot up for ifupdown... [ OK ] Finished Load/Save Random Seed. [ 19.705641] systemd-journald[366]: File /var/log/journal/5896a171b9d64a25b2db7eb12a64f597/system.journal corrupted or uncleanly shut down, renaming and replacing. [ OK ] Finished Create System Users. Starting Create Static Device Nodes in /dev... [ OK ] Finished Apply Kernel Variables. [ OK ] Finished Initial Check File System Quotas. [ OK ] Finished Flush Journal to Persistent Storage. [ OK ] Finished Create Static Device Nodes in /dev. [ OK ] Reached target Preparation for Local File Systems. [ OK ] Reached target Local File Systems. Starting Load AppArmor profiles... Starting Set console font and keymap... Starting Create final runt…dir for shutdown pivot root... Starting Tell Plymouth To Write Out Runtime Data... Starting QEMU KVM preparation - module, ksm, hugepages... Starting Create Volatile Files and Directories... Starting Rule-based Manage…for Device Events and Files... Starting Uncomplicated firewall... [ OK ] Finished Set console font and keymap. [ OK ] Finished Create final runt…e dir for shutdown pivot root. [ OK ] Finished Tell Plymouth To Write Out Runtime Data. [ OK ] Finished Uncomplicated firewall. [ OK ] Reached target Preparation for Network. [ OK ] Finished Create Volatile Files and Directories. Mounting RPC Pipe File System... Starting Load Kernel Module chromeos_pstore... Starting Load Kernel Module efi_pstore... Starting Load Kernel Module pstore_blk... Starting Load Kernel Module pstore_zone... Starting Load Kernel Module ramoops... Starting RPC bind portmap service... Starting Record System Boot/Shutdown in UTMP... [ OK ] Finished Load AppArmor profiles. [ OK ] Started Rule-based Manager for Device Events and Files. [ OK ] Mounted RPC Pipe File System. [ OK ] Finished Load Kernel Module chromeos_pstore. [ OK ] Finished Load Kernel Module efi_pstore. [ OK ] Finished QEMU KVM preparation - module, ksm, hugepages. [ OK ] Finished Load Kernel Module pstore_blk. [ OK ] Finished Load Kernel Module pstore_zone. [ OK ] Finished Load Kernel Module ramoops. [ OK ] Reached target rpc_pipefs.target. Starting pNFS block layout mapping daemon... Starting NFSv4 ID-name mapping service... Starting NFSv4 Client Tracking Daemon... [ OK ] Started Dispatch Password …ts to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Path Units. [ OK ] Reached target NFS client services. Starting Network Configuration... [ OK ] Started RPC bind portmap service. [ OK ] Finished Record System Boot/Shutdown in UTMP. [ OK ] Started pNFS block layout mapping daemon. [ OK ] Started NFSv4 ID-name mapping service. [ OK ] Started NFSv4 Client Tracking Daemon. [ OK ] Found device /dev/ttyS1. [ OK ] Found device /dev/vg_nvme/lv_5. [ OK ] Created slice Slice /system/lvm2-pvscan. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target System Initialization. [ OK ] Started Daily apt download activities. [ OK ] Started Daily apt upgrade and clean activities. [ OK ] Started Daily dpkg database backup timer. [ OK ] Started Periodic ext4 Onli…ata Check for All Filesystems. [ OK ] Started Discard unused blocks once a week. [ OK ] Started Daily rotation of log files. [ OK ] Started Daily man-db regeneration. [ OK ] Started Message of the Day. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started Ubuntu Advantage Timer for running repeated jobs. [ OK ] Reached target Timer Units. [ OK ] Listening on D-Bus System Message Bus Socket. Starting Docker Socket for the API... [ OK ] Listening on Open-iSCSI iscsid Socket. [ OK ] Listening on Load/Save RF …itch Status /dev/rfkill Watch. [ OK ] Listening on UUID daemon activation socket. Mounting /var/lib/ceph... Starting LVM event activation on device 259:0... [ OK ] Started Network Configuration. [ OK ] Finished Helper to synchronize boot up for ifupdown. [ OK ] Listening on Docker Socket for the API. [ OK ] Mounted /var/lib/ceph. [ OK ] Found device 82599ES 10-Gi…ork Connection (AOC-STGN-i2S). [ OK ] Finished LVM event activation on device 259:0. [ OK ] Reached target Socket Units. [ OK ] Reached target Basic System. Starting Accounts Service... [ OK ] Started Ceph crash dump collector. [ OK ] Reached target ceph target…h*@.service instances at once. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Save initial kernel messages after boot. Starting Remove Stale Onli…t4 Metadata Check Snapshots... Starting Record successful boot for GRUB... [ OK ] Started 1. [ OK ] Started irqbalance daemon. [ OK ] Started LTTng session daemon. Starting Dispatcher daemon for systemd-networkd... Starting Raise network interfaces... [ OK ] Started ntp-systemd-netif.service. Starting RPC Remote Quota Server... Starting System Logging Service... Starting Self Monitoring a…g Technology (SMART) Daemon... Starting User Login Management... Starting Wait for Network to be Configured... Starting Network Name Resolution... [ OK ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots. [ OK ] Started RPC Remote Quota Server. [ OK ] Finished Record successful boot for GRUB. [ OK ] Started User Login Management. Starting GRUB failed boot detection... [ OK ] Started System Logging Service. [ OK ] Finished GRUB failed boot detection. [ OK ] Started Self Monitoring an…ing Technology (SMART) Daemon. [ OK ] Started Network Name Resolution. [ OK ] Reached target Host and Network Name Lookups. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Accounts Service. [ OK ] Started Dispatcher daemon for systemd-networkd. [* ] (1 of 2) A start job is running for… network interfaces (6s / 5min 1s) M [ OK ] Finished Raise network interfaces. [ OK ] Reached target Network. Starting containerd container runtime... Starting Network Time Service... Starting OpenBSD Secure Shell server... Starting (i)SCSI target daemon... [ OK ] Started Unattended Upgrades Shutdown. [ OK ] Started Network Time Service. [ OK ] Started OpenBSD Secure Shell server. [ OK ] Started (i)SCSI target daemon. [ OK ] Started containerd container runtime. [ OK ] Started ntp-systemd-netif.service. [ OK ] Finished Wait for Network to be Configured. [ OK ] Reached target Network is Online. Starting Docker Application Container Engine... Starting NFS Mount Daemon... Starting Connect NVMe-oF s…s automatically during boot... Starting Map RBD devices... Starting /etc/rc.local Compatibility... + '[' '!' -f /.cephlab_net_configured ']' + set +e + attempts=0 + myips= + '[' '' '!=' '' ']' + '[' 0 -ge 10 ']' ++ grep -oP '(?<=inet\s)\d+(\.\d+){3}' ++ ip -4 addr ++ grep -v '127.0.0.1\|127.0.1.1' Starting NFS status monitor for NFSv2/3 locking.... Starting Ubuntu FAN network setup... [ OK ] Finished Map RBD devices. [ OK ] Finished Availability of block devices. [ OK ] Started NFS Mount Daemon. [ OK ] Started NFS status monitor for NFSv2/3 locking.. Starting NFS server and services... + myips=172.21.15.96 + attempts=1 + sleep 1 [ OK ] Finished Connect NVMe-oF s…ems automatically during boot. [ OK ] Reached target Preparation for Remote File Systems. [ OK ] Reached target Remote File Systems. Starting The Apache HTTP Server... Starting LSB: Collectl monitors system performance.... [ OK ] Started Regular background program processing daemon. [ OK ] Started Nagios Remote Plugin Executor. Starting LSB: radosgw RESTful rados gateway... Starting Permit User Sessions... [ OK ] Finished Permit User Sessions. [ OK ] Started LSB: radosgw RESTful rados gateway. [ OK ] Finished Ubuntu FAN network setup. [ OK ] Finished NFS server and services. Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. + '[' 172.21.15.96 '!=' '' ']' + set -e + '[' -n 172.21.15.96 ']' + for ip in $myips + timeout 1s ping -I 172.21.15.96 -nq -c1 172.21.0.1 ++ dig +short -x 172.21.15.96 @172.21.0.1 ++ sed 's/\.com.*/\.com/g' [ OK ] Started The Apache HTTP Server. + newhostname=smithi096.front.sepia.ceph.com + '[' -n smithi096.front.sepia.ceph.com ']' + hostname smithi096.front.sepia.ceph.com ++ hostname -d + newdomain=front.sepia.ceph.com ++ hostname -s + shorthostname=smithi096 + echo smithi096 + grep -q front.sepia.ceph.com /etc/hosts + sed -i 's/.*front.sepia.ceph.com.*/172.21.15.96 smithi096.front.sepia.ceph.com smithi096/g' /etc/hosts + break + command -v zypper + command -v apt-get + '[' '!' -f /etc/ssh/ssh_host_rsa_key ']' + '[' -e /.cephlab_rc_local ']' + exit 0 [ OK ] Started /etc/rc.local Compatibility. Starting Hold until boot process finishes up... Starting Terminate Plymouth Boot Screen... [ OK ] Finished Hold until boot process finishes up. [ OK ] Finished Terminate Plymouth Boot Screen. [ OK ] Started Serial Getty on ttyS1. Starting Set console scheme... [ OK ] Finished Set console scheme. [ OK ] Created slice Slice /system/getty. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started LSB: Collectl monitors system performance.. [ OK ] Started Docker Application Container Engine. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Record Runlevel Change in UTMP... [ OK ] Finished Record Runlevel Change in UTMP. Ubuntu 22.04.1 LTS smithi096.front.sepia.ceph.com ttyS1 smithi096 login: [disconnect] 2024-04-18T22:26:14.756 INFO:teuthology.misc:Re-opening connections... 2024-04-18T22:26:14.756 INFO:teuthology.misc:trying to connect to ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:26:14.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:26:14.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:26:14.955 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-18T22:26:15.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-18T22:26:15.411 DEBUG:teuthology.misc:waited 0.654991626739502 2024-04-18T22:26:16.412 DEBUG:tasks.ceph_manager:Mounting data for osd.1 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:26:16.412 INFO:tasks.ceph_manager:Mounting osd.1: dev: ubuntu@smithi096.front.sepia.ceph.com, cluster: cephmountpoint: /var/lib/ceph/osd/ceph-1, type: xfs, options: ['noatime'] 2024-04-18T22:26:16.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-18T22:26:16.515 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T22:26:16.535 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-18T22:26:16.536 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-18T22:26:16.578 INFO:tasks.ceph.osd.1:Started 2024-04-18T22:26:16.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:26:16.720 INFO:teuthology.orchestra.run.smithi096.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-18T22:26:16.722 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T22:26:16.723 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-04-18T22:26:17.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-18T22:26:17.731 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:26:17.700+0000 7f19006267c0 -1 Falling back to public interface 2024-04-18T22:26:21.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:no valid command found; 10 closest matches: 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:0 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:1 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:2 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:abort 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:assert 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:bluefs debug_inject_read_zeros 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:bluefs files list 2024-04-18T22:26:21.840 INFO:teuthology.orchestra.run.smithi096.stderr:bluefs stats 2024-04-18T22:26:21.841 INFO:teuthology.orchestra.run.smithi096.stderr:bluestore allocator dump block 2024-04-18T22:26:21.841 INFO:teuthology.orchestra.run.smithi096.stderr:bluestore allocator fragmentation block 2024-04-18T22:26:21.841 INFO:teuthology.orchestra.run.smithi096.stderr:admin_socket: invalid command 2024-04-18T22:26:21.842 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T22:26:21.843 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-04-18T22:26:23.905 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:26:23.872+0000 7f19006267c0 -1 osd.1 35 log_to_monitors true 2024-04-18T22:26:24.066 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-04-18T22:26:24.032+0000 7f18f4404640 -1 osd.1 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T22:26:26.844 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T22:26:26.975 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-18T22:26:26.975 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-04-18T22:26:26.975 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-04-18T22:26:26.976 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-18T22:26:31.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-18T22:26:32.141 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a: {} 2024-04-18T22:26:32.141 INFO:teuthology.orchestra.run.smithi032.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T22:26:32.150 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b: {} 2024-04-18T22:26:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T22:26:32.159 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c: {} 2024-04-18T22:26:32.159 INFO:teuthology.orchestra.run.smithi032.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T22:26:32.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-18T22:26:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.0 to 1 (10000) 2024-04-18T22:26:33.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-18T22:26:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-18T22:26:34.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-18T22:26:35.218 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 1 (10000) 2024-04-18T22:26:35.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-18T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-18T22:26:36.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-18T22:26:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.1 to 1 (10000) 2024-04-18T22:26:37.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-18T22:26:38.237 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-18T22:26:38.261 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-18T22:26:38.261 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-18T22:26:38.261 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-18T22:26:38.261 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-18T22:26:38.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-18T22:26:38.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:26:38.564 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:38.564 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:26:38.231275+0000","last_up_change":"2024-04-18T22:26:25.018371+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":35,"up_from":75,"up_thru":78,"down_at":36,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6801","nonce":1652764673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6803","nonce":1652764673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6807","nonce":1652764673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6805","nonce":1652764673}]},"public_addr":"172.21.15.96:6801/1652764673","cluster_addr":"172.21.15.96:6803/1652764673","heartbeat_back_addr":"172.21.15.96:6807/1652764673","heartbeat_front_addr":"172.21.15.96:6805/1652764673","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":35},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:26:38.584 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-18T22:26:38.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:26:38.878 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:38.879 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:26:38.231275+0000","last_up_change":"2024-04-18T22:26:25.018371+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":35,"up_from":75,"up_thru":78,"down_at":36,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6801","nonce":1652764673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6803","nonce":1652764673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6807","nonce":1652764673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6805","nonce":1652764673}]},"public_addr":"172.21.15.96:6801/1652764673","cluster_addr":"172.21.15.96:6803/1652764673","heartbeat_back_addr":"172.21.15.96:6807/1652764673","heartbeat_front_addr":"172.21.15.96:6805/1652764673","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":35},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:26:38.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-18T22:26:38.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-18T22:26:38.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-18T22:26:39.055 INFO:teuthology.orchestra.run.smithi032.stdout:322122547204 2024-04-18T22:26:39.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T22:26:39.056 INFO:teuthology.orchestra.run.smithi032.stdout:34359738422 2024-04-18T22:26:39.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T22:26:39.066 INFO:teuthology.orchestra.run.smithi032.stdout:34359738424 2024-04-18T22:26:39.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T22:26:39.349 INFO:teuthology.orchestra.run.smithi032.stdout:322122547203 2024-04-18T22:26:39.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 322122547204 got 322122547203 for osd.1 2024-04-18T22:26:39.390 INFO:teuthology.orchestra.run.smithi032.stdout:34359738421 2024-04-18T22:26:39.396 INFO:teuthology.orchestra.run.smithi032.stdout:34359738423 2024-04-18T22:26:39.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738422 got 34359738421 for osd.0 2024-04-18T22:26:39.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738424 got 34359738423 for osd.2 2024-04-18T22:26:40.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T22:26:40.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T22:26:40.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T22:26:40.692 INFO:teuthology.orchestra.run.smithi032.stdout:322122547203 2024-04-18T22:26:40.707 INFO:teuthology.orchestra.run.smithi032.stdout:34359738421 2024-04-18T22:26:40.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 322122547204 got 322122547203 for osd.1 2024-04-18T22:26:40.716 INFO:teuthology.orchestra.run.smithi032.stdout:34359738423 2024-04-18T22:26:40.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738422 got 34359738421 for osd.0 2024-04-18T22:26:40.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738424 got 34359738423 for osd.2 2024-04-18T22:26:41.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T22:26:41.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T22:26:41.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T22:26:42.025 INFO:teuthology.orchestra.run.smithi032.stdout:322122547205 2024-04-18T22:26:42.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 322122547204 got 322122547205 for osd.1 2024-04-18T22:26:42.043 DEBUG:teuthology.parallel:result is None 2024-04-18T22:26:42.046 INFO:teuthology.orchestra.run.smithi032.stdout:34359738423 2024-04-18T22:26:42.050 INFO:teuthology.orchestra.run.smithi032.stdout:34359738425 2024-04-18T22:26:42.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738422 got 34359738423 for osd.0 2024-04-18T22:26:42.066 DEBUG:teuthology.parallel:result is None 2024-04-18T22:26:42.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738424 got 34359738425 for osd.2 2024-04-18T22:26:42.068 DEBUG:teuthology.parallel:result is None 2024-04-18T22:26:42.068 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-18T22:26:42.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:26:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-18T22:26:40.948187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669824,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445696,"statfs":{"total":289910292480,"available":289224392704,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734119"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211682+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211682+0000","last_peered":"2024-04-18T22:26:38.211682+0000","last_clean":"2024-04-18T22:26:38.211682+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211682+0000","last_undegraded":"2024-04-18T22:26:38.211682+0000","last_fullsized":"2024-04-18T22:26:38.211682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:16:02.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211107+0000","last_change":"2024-04-18T22:26:26.036531+0000","last_active":"2024-04-18T22:26:38.211107+0000","last_peered":"2024-04-18T22:26:38.211107+0000","last_clean":"2024-04-18T22:26:38.211107+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:26:38.211107+0000","last_undegraded":"2024-04-18T22:26:38.211107+0000","last_fullsized":"2024-04-18T22:26:38.211107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:17:26.168838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241489+0000","last_change":"2024-04-18T22:26:37.250608+0000","last_active":"2024-04-18T22:26:38.241489+0000","last_peered":"2024-04-18T22:26:38.241489+0000","last_clean":"2024-04-18T22:26:38.241489+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:38.241489+0000","last_undegraded":"2024-04-18T22:26:38.241489+0000","last_fullsized":"2024-04-18T22:26:38.241489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:32.948871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240445+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:38.240445+0000","last_peered":"2024-04-18T22:26:38.240445+0000","last_clean":"2024-04-18T22:26:38.240445+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:38.240445+0000","last_undegraded":"2024-04-18T22:26:38.240445+0000","last_fullsized":"2024-04-18T22:26:38.240445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:26:07.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210220+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210220+0000","last_peered":"2024-04-18T22:26:38.210220+0000","last_clean":"2024-04-18T22:26:38.210220+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210220+0000","last_undegraded":"2024-04-18T22:26:38.210220+0000","last_fullsized":"2024-04-18T22:26:38.210220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:35:56.119968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239263+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239263+0000","last_peered":"2024-04-18T22:26:38.239263+0000","last_clean":"2024-04-18T22:26:38.239263+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239263+0000","last_undegraded":"2024-04-18T22:26:38.239263+0000","last_fullsized":"2024-04-18T22:26:38.239263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:04:33.941262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239350+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:38.239350+0000","last_peered":"2024-04-18T22:26:38.239350+0000","last_clean":"2024-04-18T22:26:38.239350+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:38.239350+0000","last_undegraded":"2024-04-18T22:26:38.239350+0000","last_fullsized":"2024-04-18T22:26:38.239350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:21:05.317874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238671+0000","last_change":"2024-04-18T22:26:37.251857+0000","last_active":"2024-04-18T22:26:38.238671+0000","last_peered":"2024-04-18T22:26:38.238671+0000","last_clean":"2024-04-18T22:26:38.238671+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:26:38.238671+0000","last_undegraded":"2024-04-18T22:26:38.238671+0000","last_fullsized":"2024-04-18T22:26:38.238671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:07.544858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340474+0000","last_change":"2024-04-18T22:26:26.069968+0000","last_active":"2024-04-18T22:26:38.340474+0000","last_peered":"2024-04-18T22:26:38.340474+0000","last_clean":"2024-04-18T22:26:38.340474+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:38.340474+0000","last_undegraded":"2024-04-18T22:26:38.340474+0000","last_fullsized":"2024-04-18T22:26:38.340474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:16:05.689716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211534+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211534+0000","last_peered":"2024-04-18T22:26:38.211534+0000","last_clean":"2024-04-18T22:26:38.211534+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211534+0000","last_undegraded":"2024-04-18T22:26:38.211534+0000","last_fullsized":"2024-04-18T22:26:38.211534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:43:12.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241442+0000","last_change":"2024-04-18T22:26:37.250594+0000","last_active":"2024-04-18T22:26:38.241442+0000","last_peered":"2024-04-18T22:26:38.241442+0000","last_clean":"2024-04-18T22:26:38.241442+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:38.241442+0000","last_undegraded":"2024-04-18T22:26:38.241442+0000","last_fullsized":"2024-04-18T22:26:38.241442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:31.385259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211588+0000","last_change":"2024-04-18T22:26:26.057510+0000","last_active":"2024-04-18T22:26:38.211588+0000","last_peered":"2024-04-18T22:26:38.211588+0000","last_clean":"2024-04-18T22:26:38.211588+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:38.211588+0000","last_undegraded":"2024-04-18T22:26:38.211588+0000","last_fullsized":"2024-04-18T22:26:38.211588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:10.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210691+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.210691+0000","last_peered":"2024-04-18T22:26:38.210691+0000","last_clean":"2024-04-18T22:26:38.210691+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.210691+0000","last_undegraded":"2024-04-18T22:26:38.210691+0000","last_fullsized":"2024-04-18T22:26:38.210691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:45:48.363681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239408+0000","last_change":"2024-04-18T22:26:37.253632+0000","last_active":"2024-04-18T22:26:38.239408+0000","last_peered":"2024-04-18T22:26:38.239408+0000","last_clean":"2024-04-18T22:26:38.239408+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:38.239408+0000","last_undegraded":"2024-04-18T22:26:38.239408+0000","last_fullsized":"2024-04-18T22:26:38.239408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:28:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239299+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239299+0000","last_peered":"2024-04-18T22:26:38.239299+0000","last_clean":"2024-04-18T22:26:38.239299+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239299+0000","last_undegraded":"2024-04-18T22:26:38.239299+0000","last_fullsized":"2024-04-18T22:26:38.239299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:08.716543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238847+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:38.238847+0000","last_peered":"2024-04-18T22:26:38.238847+0000","last_clean":"2024-04-18T22:26:38.238847+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:38.238847+0000","last_undegraded":"2024-04-18T22:26:38.238847+0000","last_fullsized":"2024-04-18T22:26:38.238847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:50.499991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340902+0000","last_change":"2024-04-18T22:26:26.070529+0000","last_active":"2024-04-18T22:26:38.340902+0000","last_peered":"2024-04-18T22:26:38.340902+0000","last_clean":"2024-04-18T22:26:38.340902+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:38.340902+0000","last_undegraded":"2024-04-18T22:26:38.340902+0000","last_fullsized":"2024-04-18T22:26:38.340902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:52:17.813661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210997+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.210997+0000","last_peered":"2024-04-18T22:26:38.210997+0000","last_clean":"2024-04-18T22:26:38.210997+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.210997+0000","last_undegraded":"2024-04-18T22:26:38.210997+0000","last_fullsized":"2024-04-18T22:26:38.210997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:25.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339371+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339371+0000","last_peered":"2024-04-18T22:26:38.339371+0000","last_clean":"2024-04-18T22:26:38.339371+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339371+0000","last_undegraded":"2024-04-18T22:26:38.339371+0000","last_fullsized":"2024-04-18T22:26:38.339371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:42.038434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240563+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:38.240563+0000","last_peered":"2024-04-18T22:26:38.240563+0000","last_clean":"2024-04-18T22:26:38.240563+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:38.240563+0000","last_undegraded":"2024-04-18T22:26:38.240563+0000","last_fullsized":"2024-04-18T22:26:38.240563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:51:13.814270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240871+0000","last_change":"2024-04-18T22:26:37.248780+0000","last_active":"2024-04-18T22:26:38.240871+0000","last_peered":"2024-04-18T22:26:38.240871+0000","last_clean":"2024-04-18T22:26:38.240871+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:38.240871+0000","last_undegraded":"2024-04-18T22:26:38.240871+0000","last_fullsized":"2024-04-18T22:26:38.240871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210771+0000","last_change":"2024-04-18T22:26:26.049675+0000","last_active":"2024-04-18T22:26:38.210771+0000","last_peered":"2024-04-18T22:26:38.210771+0000","last_clean":"2024-04-18T22:26:38.210771+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:38.210771+0000","last_undegraded":"2024-04-18T22:26:38.210771+0000","last_fullsized":"2024-04-18T22:26:38.210771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:27.906774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338129+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:38.338129+0000","last_peered":"2024-04-18T22:26:38.338129+0000","last_clean":"2024-04-18T22:26:38.338129+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:38.338129+0000","last_undegraded":"2024-04-18T22:26:38.338129+0000","last_fullsized":"2024-04-18T22:26:38.338129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209335+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209335+0000","last_peered":"2024-04-18T22:26:38.209335+0000","last_clean":"2024-04-18T22:26:38.209335+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209335+0000","last_undegraded":"2024-04-18T22:26:38.209335+0000","last_fullsized":"2024-04-18T22:26:38.209335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:20:10.385885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210802+0000","last_change":"2024-04-18T22:26:26.053562+0000","last_active":"2024-04-18T22:26:38.210802+0000","last_peered":"2024-04-18T22:26:38.210802+0000","last_clean":"2024-04-18T22:26:38.210802+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:38.210802+0000","last_undegraded":"2024-04-18T22:26:38.210802+0000","last_fullsized":"2024-04-18T22:26:38.210802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:03.633713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340461+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:38.340461+0000","last_peered":"2024-04-18T22:26:38.340461+0000","last_clean":"2024-04-18T22:26:38.340461+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:38.340461+0000","last_undegraded":"2024-04-18T22:26:38.340461+0000","last_fullsized":"2024-04-18T22:26:38.340461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241369+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:38.241369+0000","last_peered":"2024-04-18T22:26:38.241369+0000","last_clean":"2024-04-18T22:26:38.241369+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:38.241369+0000","last_undegraded":"2024-04-18T22:26:38.241369+0000","last_fullsized":"2024-04-18T22:26:38.241369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:07.716048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211498+0000","last_change":"2024-04-18T22:26:26.057802+0000","last_active":"2024-04-18T22:26:38.211498+0000","last_peered":"2024-04-18T22:26:38.211498+0000","last_clean":"2024-04-18T22:26:38.211498+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:26:38.211498+0000","last_undegraded":"2024-04-18T22:26:38.211498+0000","last_fullsized":"2024-04-18T22:26:38.211498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:58:25.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339284+0000","last_change":"2024-04-18T22:26:34.217013+0000","last_active":"2024-04-18T22:26:38.339284+0000","last_peered":"2024-04-18T22:26:38.339284+0000","last_clean":"2024-04-18T22:26:38.339284+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:26:38.339284+0000","last_undegraded":"2024-04-18T22:26:38.339284+0000","last_fullsized":"2024-04-18T22:26:38.339284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:37.784925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210263+0000","last_change":"2024-04-18T22:26:26.050006+0000","last_active":"2024-04-18T22:26:38.210263+0000","last_peered":"2024-04-18T22:26:38.210263+0000","last_clean":"2024-04-18T22:26:38.210263+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:38.210263+0000","last_undegraded":"2024-04-18T22:26:38.210263+0000","last_fullsized":"2024-04-18T22:26:38.210263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:33:05.635452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338277+0000","last_change":"2024-04-18T22:26:26.060702+0000","last_active":"2024-04-18T22:26:38.338277+0000","last_peered":"2024-04-18T22:26:38.338277+0000","last_clean":"2024-04-18T22:26:38.338277+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:38.338277+0000","last_undegraded":"2024-04-18T22:26:38.338277+0000","last_fullsized":"2024-04-18T22:26:38.338277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:18:36.194967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239065+0000","last_change":"2024-04-18T22:26:37.250932+0000","last_active":"2024-04-18T22:26:38.239065+0000","last_peered":"2024-04-18T22:26:38.239065+0000","last_clean":"2024-04-18T22:26:38.239065+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:38.239065+0000","last_undegraded":"2024-04-18T22:26:38.239065+0000","last_fullsized":"2024-04-18T22:26:38.239065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:03.780060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210800+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210800+0000","last_peered":"2024-04-18T22:26:38.210800+0000","last_clean":"2024-04-18T22:26:38.210800+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210800+0000","last_undegraded":"2024-04-18T22:26:38.210800+0000","last_fullsized":"2024-04-18T22:26:38.210800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:29:28.068569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241235+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:38.241235+0000","last_peered":"2024-04-18T22:26:38.241235+0000","last_clean":"2024-04-18T22:26:38.241235+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:38.241235+0000","last_undegraded":"2024-04-18T22:26:38.241235+0000","last_fullsized":"2024-04-18T22:26:38.241235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:49:04.469351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211082+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211082+0000","last_peered":"2024-04-18T22:26:38.211082+0000","last_clean":"2024-04-18T22:26:38.211082+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211082+0000","last_undegraded":"2024-04-18T22:26:38.211082+0000","last_fullsized":"2024-04-18T22:26:38.211082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:49:20.072221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339128+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:26:38.339128+0000","last_peered":"2024-04-18T22:26:38.339128+0000","last_clean":"2024-04-18T22:26:38.339128+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:26:38.339128+0000","last_undegraded":"2024-04-18T22:26:38.339128+0000","last_fullsized":"2024-04-18T22:26:38.339128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340715+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340715+0000","last_peered":"2024-04-18T22:26:38.340715+0000","last_clean":"2024-04-18T22:26:38.340715+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340715+0000","last_undegraded":"2024-04-18T22:26:38.340715+0000","last_fullsized":"2024-04-18T22:26:38.340715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209937+0000","last_change":"2024-04-18T22:26:26.052492+0000","last_active":"2024-04-18T22:26:38.209937+0000","last_peered":"2024-04-18T22:26:38.209937+0000","last_clean":"2024-04-18T22:26:38.209937+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:38.209937+0000","last_undegraded":"2024-04-18T22:26:38.209937+0000","last_fullsized":"2024-04-18T22:26:38.209937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:39:51.990646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339820+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339820+0000","last_peered":"2024-04-18T22:26:38.339820+0000","last_clean":"2024-04-18T22:26:38.339820+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339820+0000","last_undegraded":"2024-04-18T22:26:38.339820+0000","last_fullsized":"2024-04-18T22:26:38.339820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:37.891211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239177+0000","last_change":"2024-04-18T22:26:37.250746+0000","last_active":"2024-04-18T22:26:38.239177+0000","last_peered":"2024-04-18T22:26:38.239177+0000","last_clean":"2024-04-18T22:26:38.239177+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:38.239177+0000","last_undegraded":"2024-04-18T22:26:38.239177+0000","last_fullsized":"2024-04-18T22:26:38.239177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:58:41.981858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240749+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:38.240749+0000","last_peered":"2024-04-18T22:26:38.240749+0000","last_clean":"2024-04-18T22:26:38.240749+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:38.240749+0000","last_undegraded":"2024-04-18T22:26:38.240749+0000","last_fullsized":"2024-04-18T22:26:38.240749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209349+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209349+0000","last_peered":"2024-04-18T22:26:38.209349+0000","last_clean":"2024-04-18T22:26:38.209349+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209349+0000","last_undegraded":"2024-04-18T22:26:38.209349+0000","last_fullsized":"2024-04-18T22:26:38.209349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:25:14.095942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210807+0000","last_change":"2024-04-18T22:26:26.056290+0000","last_active":"2024-04-18T22:26:38.210807+0000","last_peered":"2024-04-18T22:26:38.210807+0000","last_clean":"2024-04-18T22:26:38.210807+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:38.210807+0000","last_undegraded":"2024-04-18T22:26:38.210807+0000","last_fullsized":"2024-04-18T22:26:38.210807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:33:17.740653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209496+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209496+0000","last_peered":"2024-04-18T22:26:38.209496+0000","last_clean":"2024-04-18T22:26:38.209496+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209496+0000","last_undegraded":"2024-04-18T22:26:38.209496+0000","last_fullsized":"2024-04-18T22:26:38.209496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:02:53.362524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241052+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:38.241052+0000","last_peered":"2024-04-18T22:26:38.241052+0000","last_clean":"2024-04-18T22:26:38.241052+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:38.241052+0000","last_undegraded":"2024-04-18T22:26:38.241052+0000","last_fullsized":"2024-04-18T22:26:38.241052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:26.383475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239565+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:38.239565+0000","last_peered":"2024-04-18T22:26:38.239565+0000","last_clean":"2024-04-18T22:26:38.239565+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:38.239565+0000","last_undegraded":"2024-04-18T22:26:38.239565+0000","last_fullsized":"2024-04-18T22:26:38.239565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:44:05.966703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241128+0000","last_change":"2024-04-18T22:26:37.248320+0000","last_active":"2024-04-18T22:26:38.241128+0000","last_peered":"2024-04-18T22:26:38.241128+0000","last_clean":"2024-04-18T22:26:38.241128+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:38.241128+0000","last_undegraded":"2024-04-18T22:26:38.241128+0000","last_fullsized":"2024-04-18T22:26:38.241128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:36.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339542+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:38.339542+0000","last_peered":"2024-04-18T22:26:38.339542+0000","last_clean":"2024-04-18T22:26:38.339542+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:38.339542+0000","last_undegraded":"2024-04-18T22:26:38.339542+0000","last_fullsized":"2024-04-18T22:26:38.339542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339629+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339629+0000","last_peered":"2024-04-18T22:26:38.339629+0000","last_clean":"2024-04-18T22:26:38.339629+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339629+0000","last_undegraded":"2024-04-18T22:26:38.339629+0000","last_fullsized":"2024-04-18T22:26:38.339629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:30.374649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240601+0000","last_change":"2024-04-18T22:26:37.253099+0000","last_active":"2024-04-18T22:26:38.240601+0000","last_peered":"2024-04-18T22:26:38.240601+0000","last_clean":"2024-04-18T22:26:38.240601+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:38.240601+0000","last_undegraded":"2024-04-18T22:26:38.240601+0000","last_fullsized":"2024-04-18T22:26:38.240601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:08.780794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240100+0000","last_change":"2024-04-18T22:26:37.242866+0000","last_active":"2024-04-18T22:26:38.240100+0000","last_peered":"2024-04-18T22:26:38.240100+0000","last_clean":"2024-04-18T22:26:38.240100+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:26:38.240100+0000","last_undegraded":"2024-04-18T22:26:38.240100+0000","last_fullsized":"2024-04-18T22:26:38.240100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:36.007938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240704+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:38.240704+0000","last_peered":"2024-04-18T22:26:38.240704+0000","last_clean":"2024-04-18T22:26:38.240704+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:38.240704+0000","last_undegraded":"2024-04-18T22:26:38.240704+0000","last_fullsized":"2024-04-18T22:26:38.240704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:44:13.894111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209504+0000","last_change":"2024-04-18T22:26:26.052580+0000","last_active":"2024-04-18T22:26:38.209504+0000","last_peered":"2024-04-18T22:26:38.209504+0000","last_clean":"2024-04-18T22:26:38.209504+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:38.209504+0000","last_undegraded":"2024-04-18T22:26:38.209504+0000","last_fullsized":"2024-04-18T22:26:38.209504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:22:38.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240069+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:38.240069+0000","last_peered":"2024-04-18T22:26:38.240069+0000","last_clean":"2024-04-18T22:26:38.240069+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:38.240069+0000","last_undegraded":"2024-04-18T22:26:38.240069+0000","last_fullsized":"2024-04-18T22:26:38.240069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209668+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209668+0000","last_peered":"2024-04-18T22:26:38.209668+0000","last_clean":"2024-04-18T22:26:38.209668+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209668+0000","last_undegraded":"2024-04-18T22:26:38.209668+0000","last_fullsized":"2024-04-18T22:26:38.209668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:48:30.030678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340512+0000","last_change":"2024-04-18T22:26:26.070592+0000","last_active":"2024-04-18T22:26:38.340512+0000","last_peered":"2024-04-18T22:26:38.340512+0000","last_clean":"2024-04-18T22:26:38.340512+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:38.340512+0000","last_undegraded":"2024-04-18T22:26:38.340512+0000","last_fullsized":"2024-04-18T22:26:38.340512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:10:37.345485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240135+0000","last_change":"2024-04-18T22:26:37.247379+0000","last_active":"2024-04-18T22:26:38.240135+0000","last_peered":"2024-04-18T22:26:38.240135+0000","last_clean":"2024-04-18T22:26:38.240135+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:26:38.240135+0000","last_undegraded":"2024-04-18T22:26:38.240135+0000","last_fullsized":"2024-04-18T22:26:38.240135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:31.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241409+0000","last_change":"2024-04-18T22:26:37.251309+0000","last_active":"2024-04-18T22:26:38.241409+0000","last_peered":"2024-04-18T22:26:38.241409+0000","last_clean":"2024-04-18T22:26:38.241409+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:26:38.241409+0000","last_undegraded":"2024-04-18T22:26:38.241409+0000","last_fullsized":"2024-04-18T22:26:38.241409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:55:54.687547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338304+0000","last_change":"2024-04-18T22:26:34.216163+0000","last_active":"2024-04-18T22:26:38.338304+0000","last_peered":"2024-04-18T22:26:38.338304+0000","last_clean":"2024-04-18T22:26:38.338304+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:38.338304+0000","last_undegraded":"2024-04-18T22:26:38.338304+0000","last_fullsized":"2024-04-18T22:26:38.338304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:07.705933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241207+0000","last_change":"2024-04-18T22:26:37.253789+0000","last_active":"2024-04-18T22:26:38.241207+0000","last_peered":"2024-04-18T22:26:38.241207+0000","last_clean":"2024-04-18T22:26:38.241207+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:38.241207+0000","last_undegraded":"2024-04-18T22:26:38.241207+0000","last_fullsized":"2024-04-18T22:26:38.241207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:56.318449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240074+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:38.240074+0000","last_peered":"2024-04-18T22:26:38.240074+0000","last_clean":"2024-04-18T22:26:38.240074+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:38.240074+0000","last_undegraded":"2024-04-18T22:26:38.240074+0000","last_fullsized":"2024-04-18T22:26:38.240074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:51:03.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338880+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:26:38.338880+0000","last_peered":"2024-04-18T22:26:38.338880+0000","last_clean":"2024-04-18T22:26:38.338880+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:26:38.338880+0000","last_undegraded":"2024-04-18T22:26:38.338880+0000","last_fullsized":"2024-04-18T22:26:38.338880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211260+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211260+0000","last_peered":"2024-04-18T22:26:38.211260+0000","last_clean":"2024-04-18T22:26:38.211260+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211260+0000","last_undegraded":"2024-04-18T22:26:38.211260+0000","last_fullsized":"2024-04-18T22:26:38.211260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:40.351753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339220+0000","last_change":"2024-04-18T22:26:26.067550+0000","last_active":"2024-04-18T22:26:38.339220+0000","last_peered":"2024-04-18T22:26:38.339220+0000","last_clean":"2024-04-18T22:26:38.339220+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:38.339220+0000","last_undegraded":"2024-04-18T22:26:38.339220+0000","last_fullsized":"2024-04-18T22:26:38.339220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:49:44.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210088+0000","last_change":"2024-04-18T22:26:26.058882+0000","last_active":"2024-04-18T22:26:38.210088+0000","last_peered":"2024-04-18T22:26:38.210088+0000","last_clean":"2024-04-18T22:26:38.210088+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:38.210088+0000","last_undegraded":"2024-04-18T22:26:38.210088+0000","last_fullsized":"2024-04-18T22:26:38.210088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:47:23.720729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340082+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340082+0000","last_peered":"2024-04-18T22:26:38.340082+0000","last_clean":"2024-04-18T22:26:38.340082+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340082+0000","last_undegraded":"2024-04-18T22:26:38.340082+0000","last_fullsized":"2024-04-18T22:26:38.340082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240361+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:38.240361+0000","last_peered":"2024-04-18T22:26:38.240361+0000","last_clean":"2024-04-18T22:26:38.240361+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:38.240361+0000","last_undegraded":"2024-04-18T22:26:38.240361+0000","last_fullsized":"2024-04-18T22:26:38.240361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:32.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241182+0000","last_change":"2024-04-18T22:26:37.247834+0000","last_active":"2024-04-18T22:26:38.241182+0000","last_peered":"2024-04-18T22:26:38.241182+0000","last_clean":"2024-04-18T22:26:38.241182+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:38.241182+0000","last_undegraded":"2024-04-18T22:26:38.241182+0000","last_fullsized":"2024-04-18T22:26:38.241182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:45:58.922778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339394+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339394+0000","last_peered":"2024-04-18T22:26:38.339394+0000","last_clean":"2024-04-18T22:26:38.339394+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339394+0000","last_undegraded":"2024-04-18T22:26:38.339394+0000","last_fullsized":"2024-04-18T22:26:38.339394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240376+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:38.240376+0000","last_peered":"2024-04-18T22:26:38.240376+0000","last_clean":"2024-04-18T22:26:38.240376+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:38.240376+0000","last_undegraded":"2024-04-18T22:26:38.240376+0000","last_fullsized":"2024-04-18T22:26:38.240376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:12:44.602848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340352+0000","last_change":"2024-04-18T22:26:26.068751+0000","last_active":"2024-04-18T22:26:38.340352+0000","last_peered":"2024-04-18T22:26:38.340352+0000","last_clean":"2024-04-18T22:26:38.340352+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:38.340352+0000","last_undegraded":"2024-04-18T22:26:38.340352+0000","last_fullsized":"2024-04-18T22:26:38.340352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:58.461110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210605+0000","last_change":"2024-04-18T22:26:26.050076+0000","last_active":"2024-04-18T22:26:38.210605+0000","last_peered":"2024-04-18T22:26:38.210605+0000","last_clean":"2024-04-18T22:26:38.210605+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:38.210605+0000","last_undegraded":"2024-04-18T22:26:38.210605+0000","last_fullsized":"2024-04-18T22:26:38.210605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:43:48.685468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339247+0000","last_change":"2024-04-18T22:26:34.214565+0000","last_active":"2024-04-18T22:26:38.339247+0000","last_peered":"2024-04-18T22:26:38.339247+0000","last_clean":"2024-04-18T22:26:38.339247+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:38.339247+0000","last_undegraded":"2024-04-18T22:26:38.339247+0000","last_fullsized":"2024-04-18T22:26:38.339247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:00:17.620775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211798+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211798+0000","last_peered":"2024-04-18T22:26:38.211798+0000","last_clean":"2024-04-18T22:26:38.211798+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211798+0000","last_undegraded":"2024-04-18T22:26:38.211798+0000","last_fullsized":"2024-04-18T22:26:38.211798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:41.505834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209934+0000","last_change":"2024-04-18T22:26:26.049861+0000","last_active":"2024-04-18T22:26:38.209934+0000","last_peered":"2024-04-18T22:26:38.209934+0000","last_clean":"2024-04-18T22:26:38.209934+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:38.209934+0000","last_undegraded":"2024-04-18T22:26:38.209934+0000","last_fullsized":"2024-04-18T22:26:38.209934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:36:09.243064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340992+0000","last_change":"2024-04-18T22:26:26.071169+0000","last_active":"2024-04-18T22:26:38.340992+0000","last_peered":"2024-04-18T22:26:38.340992+0000","last_clean":"2024-04-18T22:26:38.340992+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:38.340992+0000","last_undegraded":"2024-04-18T22:26:38.340992+0000","last_fullsized":"2024-04-18T22:26:38.340992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:28.018224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339892+0000","last_change":"2024-04-18T22:26:34.216022+0000","last_active":"2024-04-18T22:26:38.339892+0000","last_peered":"2024-04-18T22:26:38.339892+0000","last_clean":"2024-04-18T22:26:38.339892+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:38.339892+0000","last_undegraded":"2024-04-18T22:26:38.339892+0000","last_fullsized":"2024-04-18T22:26:38.339892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:11:39.518803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338036+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338036+0000","last_peered":"2024-04-18T22:26:38.338036+0000","last_clean":"2024-04-18T22:26:38.338036+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338036+0000","last_undegraded":"2024-04-18T22:26:38.338036+0000","last_fullsized":"2024-04-18T22:26:38.338036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:37.018548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340605+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340605+0000","last_peered":"2024-04-18T22:26:38.340605+0000","last_clean":"2024-04-18T22:26:38.340605+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340605+0000","last_undegraded":"2024-04-18T22:26:38.340605+0000","last_fullsized":"2024-04-18T22:26:38.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:31.938261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240827+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:38.240827+0000","last_peered":"2024-04-18T22:26:38.240827+0000","last_clean":"2024-04-18T22:26:38.240827+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:38.240827+0000","last_undegraded":"2024-04-18T22:26:38.240827+0000","last_fullsized":"2024-04-18T22:26:38.240827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:39.646164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210238+0000","last_change":"2024-04-18T22:26:26.049399+0000","last_active":"2024-04-18T22:26:38.210238+0000","last_peered":"2024-04-18T22:26:38.210238+0000","last_clean":"2024-04-18T22:26:38.210238+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:26:38.210238+0000","last_undegraded":"2024-04-18T22:26:38.210238+0000","last_fullsized":"2024-04-18T22:26:38.210238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:58:33.909228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240079+0000","last_change":"2024-04-18T22:26:37.253737+0000","last_active":"2024-04-18T22:26:38.240079+0000","last_peered":"2024-04-18T22:26:38.240079+0000","last_clean":"2024-04-18T22:26:38.240079+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:38.240079+0000","last_undegraded":"2024-04-18T22:26:38.240079+0000","last_fullsized":"2024-04-18T22:26:38.240079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:58:47.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240898+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.240898+0000","last_peered":"2024-04-18T22:26:38.240898+0000","last_clean":"2024-04-18T22:26:38.240898+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.240898+0000","last_undegraded":"2024-04-18T22:26:38.240898+0000","last_fullsized":"2024-04-18T22:26:38.240898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:33.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338838+0000","last_change":"2024-04-18T22:26:26.059340+0000","last_active":"2024-04-18T22:26:38.338838+0000","last_peered":"2024-04-18T22:26:38.338838+0000","last_clean":"2024-04-18T22:26:38.338838+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:26:38.338838+0000","last_undegraded":"2024-04-18T22:26:38.338838+0000","last_fullsized":"2024-04-18T22:26:38.338838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:53:02.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211394+0000","last_change":"2024-04-18T22:26:26.058539+0000","last_active":"2024-04-18T22:26:38.211394+0000","last_peered":"2024-04-18T22:26:38.211394+0000","last_clean":"2024-04-18T22:26:38.211394+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:38.211394+0000","last_undegraded":"2024-04-18T22:26:38.211394+0000","last_fullsized":"2024-04-18T22:26:38.211394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:29.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240762+0000","last_change":"2024-04-18T22:26:37.245223+0000","last_active":"2024-04-18T22:26:38.240762+0000","last_peered":"2024-04-18T22:26:38.240762+0000","last_clean":"2024-04-18T22:26:38.240762+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:26:38.240762+0000","last_undegraded":"2024-04-18T22:26:38.240762+0000","last_fullsized":"2024-04-18T22:26:38.240762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:06:32.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338415+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338415+0000","last_peered":"2024-04-18T22:26:38.338415+0000","last_clean":"2024-04-18T22:26:38.338415+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338415+0000","last_undegraded":"2024-04-18T22:26:38.338415+0000","last_fullsized":"2024-04-18T22:26:38.338415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340762+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340762+0000","last_peered":"2024-04-18T22:26:38.340762+0000","last_clean":"2024-04-18T22:26:38.340762+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340762+0000","last_undegraded":"2024-04-18T22:26:38.340762+0000","last_fullsized":"2024-04-18T22:26:38.340762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338370+0000","last_change":"2024-04-18T22:26:34.221491+0000","last_active":"2024-04-18T22:26:38.338370+0000","last_peered":"2024-04-18T22:26:38.338370+0000","last_clean":"2024-04-18T22:26:38.338370+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:38.338370+0000","last_undegraded":"2024-04-18T22:26:38.338370+0000","last_fullsized":"2024-04-18T22:26:38.338370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211095+0000","last_change":"2024-04-18T22:26:26.053516+0000","last_active":"2024-04-18T22:26:38.211095+0000","last_peered":"2024-04-18T22:26:38.211095+0000","last_clean":"2024-04-18T22:26:38.211095+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:38.211095+0000","last_undegraded":"2024-04-18T22:26:38.211095+0000","last_fullsized":"2024-04-18T22:26:38.211095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:27:51.463983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338110+0000","last_change":"2024-04-18T22:26:34.215787+0000","last_active":"2024-04-18T22:26:38.338110+0000","last_peered":"2024-04-18T22:26:38.338110+0000","last_clean":"2024-04-18T22:26:38.338110+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:38.338110+0000","last_undegraded":"2024-04-18T22:26:38.338110+0000","last_fullsized":"2024-04-18T22:26:38.338110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:32.105080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340172+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:26:38.340172+0000","last_peered":"2024-04-18T22:26:38.340172+0000","last_clean":"2024-04-18T22:26:38.340172+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:26:38.340172+0000","last_undegraded":"2024-04-18T22:26:38.340172+0000","last_fullsized":"2024-04-18T22:26:38.340172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338576+0000","last_change":"2024-04-18T22:26:34.218064+0000","last_active":"2024-04-18T22:26:38.338576+0000","last_peered":"2024-04-18T22:26:38.338576+0000","last_clean":"2024-04-18T22:26:38.338576+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:26:38.338576+0000","last_undegraded":"2024-04-18T22:26:38.338576+0000","last_fullsized":"2024-04-18T22:26:38.338576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211199+0000","last_change":"2024-04-18T22:26:26.058084+0000","last_active":"2024-04-18T22:26:38.211199+0000","last_peered":"2024-04-18T22:26:38.211199+0000","last_clean":"2024-04-18T22:26:38.211199+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:26:38.211199+0000","last_undegraded":"2024-04-18T22:26:38.211199+0000","last_fullsized":"2024-04-18T22:26:38.211199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:53:45.480811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209942+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.209942+0000","last_peered":"2024-04-18T22:26:38.209942+0000","last_clean":"2024-04-18T22:26:38.209942+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.209942+0000","last_undegraded":"2024-04-18T22:26:38.209942+0000","last_fullsized":"2024-04-18T22:26:38.209942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:06:09.947309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211063+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211063+0000","last_peered":"2024-04-18T22:26:38.211063+0000","last_clean":"2024-04-18T22:26:38.211063+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211063+0000","last_undegraded":"2024-04-18T22:26:38.211063+0000","last_fullsized":"2024-04-18T22:26:38.211063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:00.276319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209992+0000","last_change":"2024-04-18T22:26:26.049503+0000","last_active":"2024-04-18T22:26:38.209992+0000","last_peered":"2024-04-18T22:26:38.209992+0000","last_clean":"2024-04-18T22:26:38.209992+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:38.209992+0000","last_undegraded":"2024-04-18T22:26:38.209992+0000","last_fullsized":"2024-04-18T22:26:38.209992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:47:34.371634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239668+0000","last_change":"2024-04-18T22:26:37.254175+0000","last_active":"2024-04-18T22:26:38.239668+0000","last_peered":"2024-04-18T22:26:38.239668+0000","last_clean":"2024-04-18T22:26:38.239668+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:38.239668+0000","last_undegraded":"2024-04-18T22:26:38.239668+0000","last_fullsized":"2024-04-18T22:26:38.239668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:48:00.575406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240707+0000","last_change":"2024-04-18T22:26:37.253978+0000","last_active":"2024-04-18T22:26:38.240707+0000","last_peered":"2024-04-18T22:26:38.240707+0000","last_clean":"2024-04-18T22:26:38.240707+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:38.240707+0000","last_undegraded":"2024-04-18T22:26:38.240707+0000","last_fullsized":"2024-04-18T22:26:38.240707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:38.579496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241140+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:38.241140+0000","last_peered":"2024-04-18T22:26:38.241140+0000","last_clean":"2024-04-18T22:26:38.241140+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:38.241140+0000","last_undegraded":"2024-04-18T22:26:38.241140+0000","last_fullsized":"2024-04-18T22:26:38.241140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:40:37.721198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210913+0000","last_change":"2024-04-18T22:26:26.054055+0000","last_active":"2024-04-18T22:26:38.210913+0000","last_peered":"2024-04-18T22:26:38.210913+0000","last_clean":"2024-04-18T22:26:38.210913+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:38.210913+0000","last_undegraded":"2024-04-18T22:26:38.210913+0000","last_fullsized":"2024-04-18T22:26:38.210913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:13.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209807+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209807+0000","last_peered":"2024-04-18T22:26:38.209807+0000","last_clean":"2024-04-18T22:26:38.209807+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209807+0000","last_undegraded":"2024-04-18T22:26:38.209807+0000","last_fullsized":"2024-04-18T22:26:38.209807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:26.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340548+0000","last_change":"2024-04-18T22:26:26.069294+0000","last_active":"2024-04-18T22:26:38.340548+0000","last_peered":"2024-04-18T22:26:38.340548+0000","last_clean":"2024-04-18T22:26:38.340548+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:38.340548+0000","last_undegraded":"2024-04-18T22:26:38.340548+0000","last_fullsized":"2024-04-18T22:26:38.340548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:46.805559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239221+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239221+0000","last_peered":"2024-04-18T22:26:38.239221+0000","last_clean":"2024-04-18T22:26:38.239221+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239221+0000","last_undegraded":"2024-04-18T22:26:38.239221+0000","last_fullsized":"2024-04-18T22:26:38.239221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339975+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.339975+0000","last_peered":"2024-04-18T22:26:38.339975+0000","last_clean":"2024-04-18T22:26:38.339975+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.339975+0000","last_undegraded":"2024-04-18T22:26:38.339975+0000","last_fullsized":"2024-04-18T22:26:38.339975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:02.769450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738425,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078668,"statfs":{"total":96636764160,"available":96336556032,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547205,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334752,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037088,"statfs":{"total":96636764160,"available":96293978112,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:26:42.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:26:42.769 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:42.771 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:26:42.785 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-18T22:26:40.948187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669824,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445696,"statfs":{"total":289910292480,"available":289224392704,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734119"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211682+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211682+0000","last_peered":"2024-04-18T22:26:38.211682+0000","last_clean":"2024-04-18T22:26:38.211682+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211682+0000","last_undegraded":"2024-04-18T22:26:38.211682+0000","last_fullsized":"2024-04-18T22:26:38.211682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:16:02.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211107+0000","last_change":"2024-04-18T22:26:26.036531+0000","last_active":"2024-04-18T22:26:38.211107+0000","last_peered":"2024-04-18T22:26:38.211107+0000","last_clean":"2024-04-18T22:26:38.211107+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:26:38.211107+0000","last_undegraded":"2024-04-18T22:26:38.211107+0000","last_fullsized":"2024-04-18T22:26:38.211107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:17:26.168838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241489+0000","last_change":"2024-04-18T22:26:37.250608+0000","last_active":"2024-04-18T22:26:38.241489+0000","last_peered":"2024-04-18T22:26:38.241489+0000","last_clean":"2024-04-18T22:26:38.241489+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:38.241489+0000","last_undegraded":"2024-04-18T22:26:38.241489+0000","last_fullsized":"2024-04-18T22:26:38.241489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:32.948871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240445+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:38.240445+0000","last_peered":"2024-04-18T22:26:38.240445+0000","last_clean":"2024-04-18T22:26:38.240445+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:38.240445+0000","last_undegraded":"2024-04-18T22:26:38.240445+0000","last_fullsized":"2024-04-18T22:26:38.240445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:26:07.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210220+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210220+0000","last_peered":"2024-04-18T22:26:38.210220+0000","last_clean":"2024-04-18T22:26:38.210220+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210220+0000","last_undegraded":"2024-04-18T22:26:38.210220+0000","last_fullsized":"2024-04-18T22:26:38.210220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:35:56.119968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239263+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239263+0000","last_peered":"2024-04-18T22:26:38.239263+0000","last_clean":"2024-04-18T22:26:38.239263+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239263+0000","last_undegraded":"2024-04-18T22:26:38.239263+0000","last_fullsized":"2024-04-18T22:26:38.239263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:04:33.941262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239350+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:38.239350+0000","last_peered":"2024-04-18T22:26:38.239350+0000","last_clean":"2024-04-18T22:26:38.239350+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:38.239350+0000","last_undegraded":"2024-04-18T22:26:38.239350+0000","last_fullsized":"2024-04-18T22:26:38.239350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:21:05.317874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238671+0000","last_change":"2024-04-18T22:26:37.251857+0000","last_active":"2024-04-18T22:26:38.238671+0000","last_peered":"2024-04-18T22:26:38.238671+0000","last_clean":"2024-04-18T22:26:38.238671+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:26:38.238671+0000","last_undegraded":"2024-04-18T22:26:38.238671+0000","last_fullsized":"2024-04-18T22:26:38.238671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:07.544858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340474+0000","last_change":"2024-04-18T22:26:26.069968+0000","last_active":"2024-04-18T22:26:38.340474+0000","last_peered":"2024-04-18T22:26:38.340474+0000","last_clean":"2024-04-18T22:26:38.340474+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:38.340474+0000","last_undegraded":"2024-04-18T22:26:38.340474+0000","last_fullsized":"2024-04-18T22:26:38.340474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:16:05.689716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211534+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211534+0000","last_peered":"2024-04-18T22:26:38.211534+0000","last_clean":"2024-04-18T22:26:38.211534+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211534+0000","last_undegraded":"2024-04-18T22:26:38.211534+0000","last_fullsized":"2024-04-18T22:26:38.211534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:43:12.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241442+0000","last_change":"2024-04-18T22:26:37.250594+0000","last_active":"2024-04-18T22:26:38.241442+0000","last_peered":"2024-04-18T22:26:38.241442+0000","last_clean":"2024-04-18T22:26:38.241442+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:38.241442+0000","last_undegraded":"2024-04-18T22:26:38.241442+0000","last_fullsized":"2024-04-18T22:26:38.241442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:31.385259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211588+0000","last_change":"2024-04-18T22:26:26.057510+0000","last_active":"2024-04-18T22:26:38.211588+0000","last_peered":"2024-04-18T22:26:38.211588+0000","last_clean":"2024-04-18T22:26:38.211588+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:38.211588+0000","last_undegraded":"2024-04-18T22:26:38.211588+0000","last_fullsized":"2024-04-18T22:26:38.211588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:10.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210691+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.210691+0000","last_peered":"2024-04-18T22:26:38.210691+0000","last_clean":"2024-04-18T22:26:38.210691+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.210691+0000","last_undegraded":"2024-04-18T22:26:38.210691+0000","last_fullsized":"2024-04-18T22:26:38.210691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:45:48.363681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239408+0000","last_change":"2024-04-18T22:26:37.253632+0000","last_active":"2024-04-18T22:26:38.239408+0000","last_peered":"2024-04-18T22:26:38.239408+0000","last_clean":"2024-04-18T22:26:38.239408+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:38.239408+0000","last_undegraded":"2024-04-18T22:26:38.239408+0000","last_fullsized":"2024-04-18T22:26:38.239408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:28:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239299+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239299+0000","last_peered":"2024-04-18T22:26:38.239299+0000","last_clean":"2024-04-18T22:26:38.239299+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239299+0000","last_undegraded":"2024-04-18T22:26:38.239299+0000","last_fullsized":"2024-04-18T22:26:38.239299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:08.716543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238847+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:38.238847+0000","last_peered":"2024-04-18T22:26:38.238847+0000","last_clean":"2024-04-18T22:26:38.238847+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:38.238847+0000","last_undegraded":"2024-04-18T22:26:38.238847+0000","last_fullsized":"2024-04-18T22:26:38.238847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:50.499991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340902+0000","last_change":"2024-04-18T22:26:26.070529+0000","last_active":"2024-04-18T22:26:38.340902+0000","last_peered":"2024-04-18T22:26:38.340902+0000","last_clean":"2024-04-18T22:26:38.340902+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:38.340902+0000","last_undegraded":"2024-04-18T22:26:38.340902+0000","last_fullsized":"2024-04-18T22:26:38.340902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:52:17.813661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210997+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.210997+0000","last_peered":"2024-04-18T22:26:38.210997+0000","last_clean":"2024-04-18T22:26:38.210997+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.210997+0000","last_undegraded":"2024-04-18T22:26:38.210997+0000","last_fullsized":"2024-04-18T22:26:38.210997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:25.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339371+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339371+0000","last_peered":"2024-04-18T22:26:38.339371+0000","last_clean":"2024-04-18T22:26:38.339371+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339371+0000","last_undegraded":"2024-04-18T22:26:38.339371+0000","last_fullsized":"2024-04-18T22:26:38.339371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:42.038434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240563+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:38.240563+0000","last_peered":"2024-04-18T22:26:38.240563+0000","last_clean":"2024-04-18T22:26:38.240563+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:38.240563+0000","last_undegraded":"2024-04-18T22:26:38.240563+0000","last_fullsized":"2024-04-18T22:26:38.240563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:51:13.814270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240871+0000","last_change":"2024-04-18T22:26:37.248780+0000","last_active":"2024-04-18T22:26:38.240871+0000","last_peered":"2024-04-18T22:26:38.240871+0000","last_clean":"2024-04-18T22:26:38.240871+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:38.240871+0000","last_undegraded":"2024-04-18T22:26:38.240871+0000","last_fullsized":"2024-04-18T22:26:38.240871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210771+0000","last_change":"2024-04-18T22:26:26.049675+0000","last_active":"2024-04-18T22:26:38.210771+0000","last_peered":"2024-04-18T22:26:38.210771+0000","last_clean":"2024-04-18T22:26:38.210771+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:38.210771+0000","last_undegraded":"2024-04-18T22:26:38.210771+0000","last_fullsized":"2024-04-18T22:26:38.210771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:27.906774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338129+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:38.338129+0000","last_peered":"2024-04-18T22:26:38.338129+0000","last_clean":"2024-04-18T22:26:38.338129+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:38.338129+0000","last_undegraded":"2024-04-18T22:26:38.338129+0000","last_fullsized":"2024-04-18T22:26:38.338129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209335+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209335+0000","last_peered":"2024-04-18T22:26:38.209335+0000","last_clean":"2024-04-18T22:26:38.209335+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209335+0000","last_undegraded":"2024-04-18T22:26:38.209335+0000","last_fullsized":"2024-04-18T22:26:38.209335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:20:10.385885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210802+0000","last_change":"2024-04-18T22:26:26.053562+0000","last_active":"2024-04-18T22:26:38.210802+0000","last_peered":"2024-04-18T22:26:38.210802+0000","last_clean":"2024-04-18T22:26:38.210802+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:38.210802+0000","last_undegraded":"2024-04-18T22:26:38.210802+0000","last_fullsized":"2024-04-18T22:26:38.210802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:03.633713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340461+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:38.340461+0000","last_peered":"2024-04-18T22:26:38.340461+0000","last_clean":"2024-04-18T22:26:38.340461+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:38.340461+0000","last_undegraded":"2024-04-18T22:26:38.340461+0000","last_fullsized":"2024-04-18T22:26:38.340461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241369+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:38.241369+0000","last_peered":"2024-04-18T22:26:38.241369+0000","last_clean":"2024-04-18T22:26:38.241369+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:38.241369+0000","last_undegraded":"2024-04-18T22:26:38.241369+0000","last_fullsized":"2024-04-18T22:26:38.241369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:07.716048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211498+0000","last_change":"2024-04-18T22:26:26.057802+0000","last_active":"2024-04-18T22:26:38.211498+0000","last_peered":"2024-04-18T22:26:38.211498+0000","last_clean":"2024-04-18T22:26:38.211498+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:26:38.211498+0000","last_undegraded":"2024-04-18T22:26:38.211498+0000","last_fullsized":"2024-04-18T22:26:38.211498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:58:25.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339284+0000","last_change":"2024-04-18T22:26:34.217013+0000","last_active":"2024-04-18T22:26:38.339284+0000","last_peered":"2024-04-18T22:26:38.339284+0000","last_clean":"2024-04-18T22:26:38.339284+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:26:38.339284+0000","last_undegraded":"2024-04-18T22:26:38.339284+0000","last_fullsized":"2024-04-18T22:26:38.339284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:37.784925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210263+0000","last_change":"2024-04-18T22:26:26.050006+0000","last_active":"2024-04-18T22:26:38.210263+0000","last_peered":"2024-04-18T22:26:38.210263+0000","last_clean":"2024-04-18T22:26:38.210263+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:38.210263+0000","last_undegraded":"2024-04-18T22:26:38.210263+0000","last_fullsized":"2024-04-18T22:26:38.210263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:33:05.635452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338277+0000","last_change":"2024-04-18T22:26:26.060702+0000","last_active":"2024-04-18T22:26:38.338277+0000","last_peered":"2024-04-18T22:26:38.338277+0000","last_clean":"2024-04-18T22:26:38.338277+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:38.338277+0000","last_undegraded":"2024-04-18T22:26:38.338277+0000","last_fullsized":"2024-04-18T22:26:38.338277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:18:36.194967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239065+0000","last_change":"2024-04-18T22:26:37.250932+0000","last_active":"2024-04-18T22:26:38.239065+0000","last_peered":"2024-04-18T22:26:38.239065+0000","last_clean":"2024-04-18T22:26:38.239065+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:38.239065+0000","last_undegraded":"2024-04-18T22:26:38.239065+0000","last_fullsized":"2024-04-18T22:26:38.239065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:03.780060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210800+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210800+0000","last_peered":"2024-04-18T22:26:38.210800+0000","last_clean":"2024-04-18T22:26:38.210800+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210800+0000","last_undegraded":"2024-04-18T22:26:38.210800+0000","last_fullsized":"2024-04-18T22:26:38.210800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:29:28.068569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241235+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:38.241235+0000","last_peered":"2024-04-18T22:26:38.241235+0000","last_clean":"2024-04-18T22:26:38.241235+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:38.241235+0000","last_undegraded":"2024-04-18T22:26:38.241235+0000","last_fullsized":"2024-04-18T22:26:38.241235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:49:04.469351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211082+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211082+0000","last_peered":"2024-04-18T22:26:38.211082+0000","last_clean":"2024-04-18T22:26:38.211082+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211082+0000","last_undegraded":"2024-04-18T22:26:38.211082+0000","last_fullsized":"2024-04-18T22:26:38.211082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:49:20.072221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339128+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:26:38.339128+0000","last_peered":"2024-04-18T22:26:38.339128+0000","last_clean":"2024-04-18T22:26:38.339128+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:26:38.339128+0000","last_undegraded":"2024-04-18T22:26:38.339128+0000","last_fullsized":"2024-04-18T22:26:38.339128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340715+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340715+0000","last_peered":"2024-04-18T22:26:38.340715+0000","last_clean":"2024-04-18T22:26:38.340715+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340715+0000","last_undegraded":"2024-04-18T22:26:38.340715+0000","last_fullsized":"2024-04-18T22:26:38.340715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209937+0000","last_change":"2024-04-18T22:26:26.052492+0000","last_active":"2024-04-18T22:26:38.209937+0000","last_peered":"2024-04-18T22:26:38.209937+0000","last_clean":"2024-04-18T22:26:38.209937+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:38.209937+0000","last_undegraded":"2024-04-18T22:26:38.209937+0000","last_fullsized":"2024-04-18T22:26:38.209937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:39:51.990646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339820+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339820+0000","last_peered":"2024-04-18T22:26:38.339820+0000","last_clean":"2024-04-18T22:26:38.339820+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339820+0000","last_undegraded":"2024-04-18T22:26:38.339820+0000","last_fullsized":"2024-04-18T22:26:38.339820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:37.891211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239177+0000","last_change":"2024-04-18T22:26:37.250746+0000","last_active":"2024-04-18T22:26:38.239177+0000","last_peered":"2024-04-18T22:26:38.239177+0000","last_clean":"2024-04-18T22:26:38.239177+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:38.239177+0000","last_undegraded":"2024-04-18T22:26:38.239177+0000","last_fullsized":"2024-04-18T22:26:38.239177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:58:41.981858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240749+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:38.240749+0000","last_peered":"2024-04-18T22:26:38.240749+0000","last_clean":"2024-04-18T22:26:38.240749+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:38.240749+0000","last_undegraded":"2024-04-18T22:26:38.240749+0000","last_fullsized":"2024-04-18T22:26:38.240749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209349+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209349+0000","last_peered":"2024-04-18T22:26:38.209349+0000","last_clean":"2024-04-18T22:26:38.209349+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209349+0000","last_undegraded":"2024-04-18T22:26:38.209349+0000","last_fullsized":"2024-04-18T22:26:38.209349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:25:14.095942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210807+0000","last_change":"2024-04-18T22:26:26.056290+0000","last_active":"2024-04-18T22:26:38.210807+0000","last_peered":"2024-04-18T22:26:38.210807+0000","last_clean":"2024-04-18T22:26:38.210807+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:38.210807+0000","last_undegraded":"2024-04-18T22:26:38.210807+0000","last_fullsized":"2024-04-18T22:26:38.210807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:33:17.740653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209496+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209496+0000","last_peered":"2024-04-18T22:26:38.209496+0000","last_clean":"2024-04-18T22:26:38.209496+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209496+0000","last_undegraded":"2024-04-18T22:26:38.209496+0000","last_fullsized":"2024-04-18T22:26:38.209496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:02:53.362524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241052+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:38.241052+0000","last_peered":"2024-04-18T22:26:38.241052+0000","last_clean":"2024-04-18T22:26:38.241052+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:38.241052+0000","last_undegraded":"2024-04-18T22:26:38.241052+0000","last_fullsized":"2024-04-18T22:26:38.241052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:26.383475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239565+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:38.239565+0000","last_peered":"2024-04-18T22:26:38.239565+0000","last_clean":"2024-04-18T22:26:38.239565+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:38.239565+0000","last_undegraded":"2024-04-18T22:26:38.239565+0000","last_fullsized":"2024-04-18T22:26:38.239565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:44:05.966703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241128+0000","last_change":"2024-04-18T22:26:37.248320+0000","last_active":"2024-04-18T22:26:38.241128+0000","last_peered":"2024-04-18T22:26:38.241128+0000","last_clean":"2024-04-18T22:26:38.241128+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:38.241128+0000","last_undegraded":"2024-04-18T22:26:38.241128+0000","last_fullsized":"2024-04-18T22:26:38.241128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:36.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339542+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:38.339542+0000","last_peered":"2024-04-18T22:26:38.339542+0000","last_clean":"2024-04-18T22:26:38.339542+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:38.339542+0000","last_undegraded":"2024-04-18T22:26:38.339542+0000","last_fullsized":"2024-04-18T22:26:38.339542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339629+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339629+0000","last_peered":"2024-04-18T22:26:38.339629+0000","last_clean":"2024-04-18T22:26:38.339629+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339629+0000","last_undegraded":"2024-04-18T22:26:38.339629+0000","last_fullsized":"2024-04-18T22:26:38.339629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:30.374649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240601+0000","last_change":"2024-04-18T22:26:37.253099+0000","last_active":"2024-04-18T22:26:38.240601+0000","last_peered":"2024-04-18T22:26:38.240601+0000","last_clean":"2024-04-18T22:26:38.240601+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:38.240601+0000","last_undegraded":"2024-04-18T22:26:38.240601+0000","last_fullsized":"2024-04-18T22:26:38.240601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:08.780794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240100+0000","last_change":"2024-04-18T22:26:37.242866+0000","last_active":"2024-04-18T22:26:38.240100+0000","last_peered":"2024-04-18T22:26:38.240100+0000","last_clean":"2024-04-18T22:26:38.240100+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:26:38.240100+0000","last_undegraded":"2024-04-18T22:26:38.240100+0000","last_fullsized":"2024-04-18T22:26:38.240100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:36.007938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240704+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:38.240704+0000","last_peered":"2024-04-18T22:26:38.240704+0000","last_clean":"2024-04-18T22:26:38.240704+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:38.240704+0000","last_undegraded":"2024-04-18T22:26:38.240704+0000","last_fullsized":"2024-04-18T22:26:38.240704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:44:13.894111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209504+0000","last_change":"2024-04-18T22:26:26.052580+0000","last_active":"2024-04-18T22:26:38.209504+0000","last_peered":"2024-04-18T22:26:38.209504+0000","last_clean":"2024-04-18T22:26:38.209504+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:38.209504+0000","last_undegraded":"2024-04-18T22:26:38.209504+0000","last_fullsized":"2024-04-18T22:26:38.209504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:22:38.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240069+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:38.240069+0000","last_peered":"2024-04-18T22:26:38.240069+0000","last_clean":"2024-04-18T22:26:38.240069+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:38.240069+0000","last_undegraded":"2024-04-18T22:26:38.240069+0000","last_fullsized":"2024-04-18T22:26:38.240069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209668+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209668+0000","last_peered":"2024-04-18T22:26:38.209668+0000","last_clean":"2024-04-18T22:26:38.209668+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209668+0000","last_undegraded":"2024-04-18T22:26:38.209668+0000","last_fullsized":"2024-04-18T22:26:38.209668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:48:30.030678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340512+0000","last_change":"2024-04-18T22:26:26.070592+0000","last_active":"2024-04-18T22:26:38.340512+0000","last_peered":"2024-04-18T22:26:38.340512+0000","last_clean":"2024-04-18T22:26:38.340512+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:38.340512+0000","last_undegraded":"2024-04-18T22:26:38.340512+0000","last_fullsized":"2024-04-18T22:26:38.340512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:10:37.345485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240135+0000","last_change":"2024-04-18T22:26:37.247379+0000","last_active":"2024-04-18T22:26:38.240135+0000","last_peered":"2024-04-18T22:26:38.240135+0000","last_clean":"2024-04-18T22:26:38.240135+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:26:38.240135+0000","last_undegraded":"2024-04-18T22:26:38.240135+0000","last_fullsized":"2024-04-18T22:26:38.240135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:31.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241409+0000","last_change":"2024-04-18T22:26:37.251309+0000","last_active":"2024-04-18T22:26:38.241409+0000","last_peered":"2024-04-18T22:26:38.241409+0000","last_clean":"2024-04-18T22:26:38.241409+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:26:38.241409+0000","last_undegraded":"2024-04-18T22:26:38.241409+0000","last_fullsized":"2024-04-18T22:26:38.241409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:55:54.687547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338304+0000","last_change":"2024-04-18T22:26:34.216163+0000","last_active":"2024-04-18T22:26:38.338304+0000","last_peered":"2024-04-18T22:26:38.338304+0000","last_clean":"2024-04-18T22:26:38.338304+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:38.338304+0000","last_undegraded":"2024-04-18T22:26:38.338304+0000","last_fullsized":"2024-04-18T22:26:38.338304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:07.705933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241207+0000","last_change":"2024-04-18T22:26:37.253789+0000","last_active":"2024-04-18T22:26:38.241207+0000","last_peered":"2024-04-18T22:26:38.241207+0000","last_clean":"2024-04-18T22:26:38.241207+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:38.241207+0000","last_undegraded":"2024-04-18T22:26:38.241207+0000","last_fullsized":"2024-04-18T22:26:38.241207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:56.318449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240074+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:38.240074+0000","last_peered":"2024-04-18T22:26:38.240074+0000","last_clean":"2024-04-18T22:26:38.240074+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:38.240074+0000","last_undegraded":"2024-04-18T22:26:38.240074+0000","last_fullsized":"2024-04-18T22:26:38.240074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:51:03.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338880+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:26:38.338880+0000","last_peered":"2024-04-18T22:26:38.338880+0000","last_clean":"2024-04-18T22:26:38.338880+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:26:38.338880+0000","last_undegraded":"2024-04-18T22:26:38.338880+0000","last_fullsized":"2024-04-18T22:26:38.338880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211260+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211260+0000","last_peered":"2024-04-18T22:26:38.211260+0000","last_clean":"2024-04-18T22:26:38.211260+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211260+0000","last_undegraded":"2024-04-18T22:26:38.211260+0000","last_fullsized":"2024-04-18T22:26:38.211260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:40.351753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339220+0000","last_change":"2024-04-18T22:26:26.067550+0000","last_active":"2024-04-18T22:26:38.339220+0000","last_peered":"2024-04-18T22:26:38.339220+0000","last_clean":"2024-04-18T22:26:38.339220+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:38.339220+0000","last_undegraded":"2024-04-18T22:26:38.339220+0000","last_fullsized":"2024-04-18T22:26:38.339220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:49:44.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210088+0000","last_change":"2024-04-18T22:26:26.058882+0000","last_active":"2024-04-18T22:26:38.210088+0000","last_peered":"2024-04-18T22:26:38.210088+0000","last_clean":"2024-04-18T22:26:38.210088+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:38.210088+0000","last_undegraded":"2024-04-18T22:26:38.210088+0000","last_fullsized":"2024-04-18T22:26:38.210088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:47:23.720729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340082+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340082+0000","last_peered":"2024-04-18T22:26:38.340082+0000","last_clean":"2024-04-18T22:26:38.340082+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340082+0000","last_undegraded":"2024-04-18T22:26:38.340082+0000","last_fullsized":"2024-04-18T22:26:38.340082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240361+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:38.240361+0000","last_peered":"2024-04-18T22:26:38.240361+0000","last_clean":"2024-04-18T22:26:38.240361+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:38.240361+0000","last_undegraded":"2024-04-18T22:26:38.240361+0000","last_fullsized":"2024-04-18T22:26:38.240361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:32.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241182+0000","last_change":"2024-04-18T22:26:37.247834+0000","last_active":"2024-04-18T22:26:38.241182+0000","last_peered":"2024-04-18T22:26:38.241182+0000","last_clean":"2024-04-18T22:26:38.241182+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:38.241182+0000","last_undegraded":"2024-04-18T22:26:38.241182+0000","last_fullsized":"2024-04-18T22:26:38.241182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:45:58.922778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339394+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339394+0000","last_peered":"2024-04-18T22:26:38.339394+0000","last_clean":"2024-04-18T22:26:38.339394+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339394+0000","last_undegraded":"2024-04-18T22:26:38.339394+0000","last_fullsized":"2024-04-18T22:26:38.339394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240376+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:38.240376+0000","last_peered":"2024-04-18T22:26:38.240376+0000","last_clean":"2024-04-18T22:26:38.240376+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:38.240376+0000","last_undegraded":"2024-04-18T22:26:38.240376+0000","last_fullsized":"2024-04-18T22:26:38.240376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:12:44.602848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340352+0000","last_change":"2024-04-18T22:26:26.068751+0000","last_active":"2024-04-18T22:26:38.340352+0000","last_peered":"2024-04-18T22:26:38.340352+0000","last_clean":"2024-04-18T22:26:38.340352+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:38.340352+0000","last_undegraded":"2024-04-18T22:26:38.340352+0000","last_fullsized":"2024-04-18T22:26:38.340352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:58.461110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210605+0000","last_change":"2024-04-18T22:26:26.050076+0000","last_active":"2024-04-18T22:26:38.210605+0000","last_peered":"2024-04-18T22:26:38.210605+0000","last_clean":"2024-04-18T22:26:38.210605+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:38.210605+0000","last_undegraded":"2024-04-18T22:26:38.210605+0000","last_fullsized":"2024-04-18T22:26:38.210605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:43:48.685468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339247+0000","last_change":"2024-04-18T22:26:34.214565+0000","last_active":"2024-04-18T22:26:38.339247+0000","last_peered":"2024-04-18T22:26:38.339247+0000","last_clean":"2024-04-18T22:26:38.339247+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:38.339247+0000","last_undegraded":"2024-04-18T22:26:38.339247+0000","last_fullsized":"2024-04-18T22:26:38.339247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:00:17.620775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211798+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211798+0000","last_peered":"2024-04-18T22:26:38.211798+0000","last_clean":"2024-04-18T22:26:38.211798+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211798+0000","last_undegraded":"2024-04-18T22:26:38.211798+0000","last_fullsized":"2024-04-18T22:26:38.211798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:41.505834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209934+0000","last_change":"2024-04-18T22:26:26.049861+0000","last_active":"2024-04-18T22:26:38.209934+0000","last_peered":"2024-04-18T22:26:38.209934+0000","last_clean":"2024-04-18T22:26:38.209934+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:38.209934+0000","last_undegraded":"2024-04-18T22:26:38.209934+0000","last_fullsized":"2024-04-18T22:26:38.209934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:36:09.243064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340992+0000","last_change":"2024-04-18T22:26:26.071169+0000","last_active":"2024-04-18T22:26:38.340992+0000","last_peered":"2024-04-18T22:26:38.340992+0000","last_clean":"2024-04-18T22:26:38.340992+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:38.340992+0000","last_undegraded":"2024-04-18T22:26:38.340992+0000","last_fullsized":"2024-04-18T22:26:38.340992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:28.018224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339892+0000","last_change":"2024-04-18T22:26:34.216022+0000","last_active":"2024-04-18T22:26:38.339892+0000","last_peered":"2024-04-18T22:26:38.339892+0000","last_clean":"2024-04-18T22:26:38.339892+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:38.339892+0000","last_undegraded":"2024-04-18T22:26:38.339892+0000","last_fullsized":"2024-04-18T22:26:38.339892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:11:39.518803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338036+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338036+0000","last_peered":"2024-04-18T22:26:38.338036+0000","last_clean":"2024-04-18T22:26:38.338036+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338036+0000","last_undegraded":"2024-04-18T22:26:38.338036+0000","last_fullsized":"2024-04-18T22:26:38.338036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:37.018548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340605+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340605+0000","last_peered":"2024-04-18T22:26:38.340605+0000","last_clean":"2024-04-18T22:26:38.340605+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340605+0000","last_undegraded":"2024-04-18T22:26:38.340605+0000","last_fullsized":"2024-04-18T22:26:38.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:31.938261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240827+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:38.240827+0000","last_peered":"2024-04-18T22:26:38.240827+0000","last_clean":"2024-04-18T22:26:38.240827+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:38.240827+0000","last_undegraded":"2024-04-18T22:26:38.240827+0000","last_fullsized":"2024-04-18T22:26:38.240827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:39.646164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210238+0000","last_change":"2024-04-18T22:26:26.049399+0000","last_active":"2024-04-18T22:26:38.210238+0000","last_peered":"2024-04-18T22:26:38.210238+0000","last_clean":"2024-04-18T22:26:38.210238+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:26:38.210238+0000","last_undegraded":"2024-04-18T22:26:38.210238+0000","last_fullsized":"2024-04-18T22:26:38.210238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:58:33.909228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240079+0000","last_change":"2024-04-18T22:26:37.253737+0000","last_active":"2024-04-18T22:26:38.240079+0000","last_peered":"2024-04-18T22:26:38.240079+0000","last_clean":"2024-04-18T22:26:38.240079+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:38.240079+0000","last_undegraded":"2024-04-18T22:26:38.240079+0000","last_fullsized":"2024-04-18T22:26:38.240079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:58:47.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240898+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.240898+0000","last_peered":"2024-04-18T22:26:38.240898+0000","last_clean":"2024-04-18T22:26:38.240898+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.240898+0000","last_undegraded":"2024-04-18T22:26:38.240898+0000","last_fullsized":"2024-04-18T22:26:38.240898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:33.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338838+0000","last_change":"2024-04-18T22:26:26.059340+0000","last_active":"2024-04-18T22:26:38.338838+0000","last_peered":"2024-04-18T22:26:38.338838+0000","last_clean":"2024-04-18T22:26:38.338838+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:26:38.338838+0000","last_undegraded":"2024-04-18T22:26:38.338838+0000","last_fullsized":"2024-04-18T22:26:38.338838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:53:02.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211394+0000","last_change":"2024-04-18T22:26:26.058539+0000","last_active":"2024-04-18T22:26:38.211394+0000","last_peered":"2024-04-18T22:26:38.211394+0000","last_clean":"2024-04-18T22:26:38.211394+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:38.211394+0000","last_undegraded":"2024-04-18T22:26:38.211394+0000","last_fullsized":"2024-04-18T22:26:38.211394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:29.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240762+0000","last_change":"2024-04-18T22:26:37.245223+0000","last_active":"2024-04-18T22:26:38.240762+0000","last_peered":"2024-04-18T22:26:38.240762+0000","last_clean":"2024-04-18T22:26:38.240762+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:26:38.240762+0000","last_undegraded":"2024-04-18T22:26:38.240762+0000","last_fullsized":"2024-04-18T22:26:38.240762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:06:32.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338415+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338415+0000","last_peered":"2024-04-18T22:26:38.338415+0000","last_clean":"2024-04-18T22:26:38.338415+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338415+0000","last_undegraded":"2024-04-18T22:26:38.338415+0000","last_fullsized":"2024-04-18T22:26:38.338415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340762+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340762+0000","last_peered":"2024-04-18T22:26:38.340762+0000","last_clean":"2024-04-18T22:26:38.340762+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340762+0000","last_undegraded":"2024-04-18T22:26:38.340762+0000","last_fullsized":"2024-04-18T22:26:38.340762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338370+0000","last_change":"2024-04-18T22:26:34.221491+0000","last_active":"2024-04-18T22:26:38.338370+0000","last_peered":"2024-04-18T22:26:38.338370+0000","last_clean":"2024-04-18T22:26:38.338370+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:38.338370+0000","last_undegraded":"2024-04-18T22:26:38.338370+0000","last_fullsized":"2024-04-18T22:26:38.338370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211095+0000","last_change":"2024-04-18T22:26:26.053516+0000","last_active":"2024-04-18T22:26:38.211095+0000","last_peered":"2024-04-18T22:26:38.211095+0000","last_clean":"2024-04-18T22:26:38.211095+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:38.211095+0000","last_undegraded":"2024-04-18T22:26:38.211095+0000","last_fullsized":"2024-04-18T22:26:38.211095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:27:51.463983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338110+0000","last_change":"2024-04-18T22:26:34.215787+0000","last_active":"2024-04-18T22:26:38.338110+0000","last_peered":"2024-04-18T22:26:38.338110+0000","last_clean":"2024-04-18T22:26:38.338110+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:38.338110+0000","last_undegraded":"2024-04-18T22:26:38.338110+0000","last_fullsized":"2024-04-18T22:26:38.338110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:32.105080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340172+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:26:38.340172+0000","last_peered":"2024-04-18T22:26:38.340172+0000","last_clean":"2024-04-18T22:26:38.340172+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:26:38.340172+0000","last_undegraded":"2024-04-18T22:26:38.340172+0000","last_fullsized":"2024-04-18T22:26:38.340172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338576+0000","last_change":"2024-04-18T22:26:34.218064+0000","last_active":"2024-04-18T22:26:38.338576+0000","last_peered":"2024-04-18T22:26:38.338576+0000","last_clean":"2024-04-18T22:26:38.338576+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:26:38.338576+0000","last_undegraded":"2024-04-18T22:26:38.338576+0000","last_fullsized":"2024-04-18T22:26:38.338576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211199+0000","last_change":"2024-04-18T22:26:26.058084+0000","last_active":"2024-04-18T22:26:38.211199+0000","last_peered":"2024-04-18T22:26:38.211199+0000","last_clean":"2024-04-18T22:26:38.211199+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:26:38.211199+0000","last_undegraded":"2024-04-18T22:26:38.211199+0000","last_fullsized":"2024-04-18T22:26:38.211199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:53:45.480811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209942+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.209942+0000","last_peered":"2024-04-18T22:26:38.209942+0000","last_clean":"2024-04-18T22:26:38.209942+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.209942+0000","last_undegraded":"2024-04-18T22:26:38.209942+0000","last_fullsized":"2024-04-18T22:26:38.209942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:06:09.947309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211063+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211063+0000","last_peered":"2024-04-18T22:26:38.211063+0000","last_clean":"2024-04-18T22:26:38.211063+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211063+0000","last_undegraded":"2024-04-18T22:26:38.211063+0000","last_fullsized":"2024-04-18T22:26:38.211063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:00.276319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209992+0000","last_change":"2024-04-18T22:26:26.049503+0000","last_active":"2024-04-18T22:26:38.209992+0000","last_peered":"2024-04-18T22:26:38.209992+0000","last_clean":"2024-04-18T22:26:38.209992+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:38.209992+0000","last_undegraded":"2024-04-18T22:26:38.209992+0000","last_fullsized":"2024-04-18T22:26:38.209992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:47:34.371634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239668+0000","last_change":"2024-04-18T22:26:37.254175+0000","last_active":"2024-04-18T22:26:38.239668+0000","last_peered":"2024-04-18T22:26:38.239668+0000","last_clean":"2024-04-18T22:26:38.239668+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:38.239668+0000","last_undegraded":"2024-04-18T22:26:38.239668+0000","last_fullsized":"2024-04-18T22:26:38.239668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:48:00.575406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240707+0000","last_change":"2024-04-18T22:26:37.253978+0000","last_active":"2024-04-18T22:26:38.240707+0000","last_peered":"2024-04-18T22:26:38.240707+0000","last_clean":"2024-04-18T22:26:38.240707+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:38.240707+0000","last_undegraded":"2024-04-18T22:26:38.240707+0000","last_fullsized":"2024-04-18T22:26:38.240707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:38.579496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241140+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:38.241140+0000","last_peered":"2024-04-18T22:26:38.241140+0000","last_clean":"2024-04-18T22:26:38.241140+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:38.241140+0000","last_undegraded":"2024-04-18T22:26:38.241140+0000","last_fullsized":"2024-04-18T22:26:38.241140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:40:37.721198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210913+0000","last_change":"2024-04-18T22:26:26.054055+0000","last_active":"2024-04-18T22:26:38.210913+0000","last_peered":"2024-04-18T22:26:38.210913+0000","last_clean":"2024-04-18T22:26:38.210913+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:38.210913+0000","last_undegraded":"2024-04-18T22:26:38.210913+0000","last_fullsized":"2024-04-18T22:26:38.210913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:13.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209807+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209807+0000","last_peered":"2024-04-18T22:26:38.209807+0000","last_clean":"2024-04-18T22:26:38.209807+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209807+0000","last_undegraded":"2024-04-18T22:26:38.209807+0000","last_fullsized":"2024-04-18T22:26:38.209807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:26.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340548+0000","last_change":"2024-04-18T22:26:26.069294+0000","last_active":"2024-04-18T22:26:38.340548+0000","last_peered":"2024-04-18T22:26:38.340548+0000","last_clean":"2024-04-18T22:26:38.340548+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:38.340548+0000","last_undegraded":"2024-04-18T22:26:38.340548+0000","last_fullsized":"2024-04-18T22:26:38.340548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:46.805559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239221+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239221+0000","last_peered":"2024-04-18T22:26:38.239221+0000","last_clean":"2024-04-18T22:26:38.239221+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239221+0000","last_undegraded":"2024-04-18T22:26:38.239221+0000","last_fullsized":"2024-04-18T22:26:38.239221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339975+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.339975+0000","last_peered":"2024-04-18T22:26:38.339975+0000","last_clean":"2024-04-18T22:26:38.339975+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.339975+0000","last_undegraded":"2024-04-18T22:26:38.339975+0000","last_fullsized":"2024-04-18T22:26:38.339975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:02.769450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738425,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078668,"statfs":{"total":96636764160,"available":96336556032,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547205,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334752,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037088,"statfs":{"total":96636764160,"available":96293978112,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:26:42.792 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-18T22:26:42.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-18T22:26:43.162 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-18T22:26:43.172 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-18T22:26:43.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:26:43.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:26:43.493 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-04-18T22:26:42.949070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669824,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445696,"statfs":{"total":289910292480,"available":289224392704,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.732639"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211682+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211682+0000","last_peered":"2024-04-18T22:26:38.211682+0000","last_clean":"2024-04-18T22:26:38.211682+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211682+0000","last_undegraded":"2024-04-18T22:26:38.211682+0000","last_fullsized":"2024-04-18T22:26:38.211682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:16:02.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211107+0000","last_change":"2024-04-18T22:26:26.036531+0000","last_active":"2024-04-18T22:26:38.211107+0000","last_peered":"2024-04-18T22:26:38.211107+0000","last_clean":"2024-04-18T22:26:38.211107+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:26:38.211107+0000","last_undegraded":"2024-04-18T22:26:38.211107+0000","last_fullsized":"2024-04-18T22:26:38.211107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:17:26.168838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241489+0000","last_change":"2024-04-18T22:26:37.250608+0000","last_active":"2024-04-18T22:26:38.241489+0000","last_peered":"2024-04-18T22:26:38.241489+0000","last_clean":"2024-04-18T22:26:38.241489+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:38.241489+0000","last_undegraded":"2024-04-18T22:26:38.241489+0000","last_fullsized":"2024-04-18T22:26:38.241489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:32.948871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240445+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:38.240445+0000","last_peered":"2024-04-18T22:26:38.240445+0000","last_clean":"2024-04-18T22:26:38.240445+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:38.240445+0000","last_undegraded":"2024-04-18T22:26:38.240445+0000","last_fullsized":"2024-04-18T22:26:38.240445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:26:07.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210220+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210220+0000","last_peered":"2024-04-18T22:26:38.210220+0000","last_clean":"2024-04-18T22:26:38.210220+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210220+0000","last_undegraded":"2024-04-18T22:26:38.210220+0000","last_fullsized":"2024-04-18T22:26:38.210220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:35:56.119968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239263+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239263+0000","last_peered":"2024-04-18T22:26:38.239263+0000","last_clean":"2024-04-18T22:26:38.239263+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239263+0000","last_undegraded":"2024-04-18T22:26:38.239263+0000","last_fullsized":"2024-04-18T22:26:38.239263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:04:33.941262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239350+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:38.239350+0000","last_peered":"2024-04-18T22:26:38.239350+0000","last_clean":"2024-04-18T22:26:38.239350+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:38.239350+0000","last_undegraded":"2024-04-18T22:26:38.239350+0000","last_fullsized":"2024-04-18T22:26:38.239350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:21:05.317874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238671+0000","last_change":"2024-04-18T22:26:37.251857+0000","last_active":"2024-04-18T22:26:38.238671+0000","last_peered":"2024-04-18T22:26:38.238671+0000","last_clean":"2024-04-18T22:26:38.238671+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:26:38.238671+0000","last_undegraded":"2024-04-18T22:26:38.238671+0000","last_fullsized":"2024-04-18T22:26:38.238671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:07.544858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340474+0000","last_change":"2024-04-18T22:26:26.069968+0000","last_active":"2024-04-18T22:26:38.340474+0000","last_peered":"2024-04-18T22:26:38.340474+0000","last_clean":"2024-04-18T22:26:38.340474+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:38.340474+0000","last_undegraded":"2024-04-18T22:26:38.340474+0000","last_fullsized":"2024-04-18T22:26:38.340474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:16:05.689716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211534+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211534+0000","last_peered":"2024-04-18T22:26:38.211534+0000","last_clean":"2024-04-18T22:26:38.211534+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211534+0000","last_undegraded":"2024-04-18T22:26:38.211534+0000","last_fullsized":"2024-04-18T22:26:38.211534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:43:12.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241442+0000","last_change":"2024-04-18T22:26:37.250594+0000","last_active":"2024-04-18T22:26:38.241442+0000","last_peered":"2024-04-18T22:26:38.241442+0000","last_clean":"2024-04-18T22:26:38.241442+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:38.241442+0000","last_undegraded":"2024-04-18T22:26:38.241442+0000","last_fullsized":"2024-04-18T22:26:38.241442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:31.385259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211588+0000","last_change":"2024-04-18T22:26:26.057510+0000","last_active":"2024-04-18T22:26:38.211588+0000","last_peered":"2024-04-18T22:26:38.211588+0000","last_clean":"2024-04-18T22:26:38.211588+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:38.211588+0000","last_undegraded":"2024-04-18T22:26:38.211588+0000","last_fullsized":"2024-04-18T22:26:38.211588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:10.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210691+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.210691+0000","last_peered":"2024-04-18T22:26:38.210691+0000","last_clean":"2024-04-18T22:26:38.210691+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.210691+0000","last_undegraded":"2024-04-18T22:26:38.210691+0000","last_fullsized":"2024-04-18T22:26:38.210691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:45:48.363681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239408+0000","last_change":"2024-04-18T22:26:37.253632+0000","last_active":"2024-04-18T22:26:38.239408+0000","last_peered":"2024-04-18T22:26:38.239408+0000","last_clean":"2024-04-18T22:26:38.239408+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:38.239408+0000","last_undegraded":"2024-04-18T22:26:38.239408+0000","last_fullsized":"2024-04-18T22:26:38.239408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:28:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239299+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239299+0000","last_peered":"2024-04-18T22:26:38.239299+0000","last_clean":"2024-04-18T22:26:38.239299+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239299+0000","last_undegraded":"2024-04-18T22:26:38.239299+0000","last_fullsized":"2024-04-18T22:26:38.239299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:08.716543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238847+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:38.238847+0000","last_peered":"2024-04-18T22:26:38.238847+0000","last_clean":"2024-04-18T22:26:38.238847+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:38.238847+0000","last_undegraded":"2024-04-18T22:26:38.238847+0000","last_fullsized":"2024-04-18T22:26:38.238847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:50.499991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340902+0000","last_change":"2024-04-18T22:26:26.070529+0000","last_active":"2024-04-18T22:26:38.340902+0000","last_peered":"2024-04-18T22:26:38.340902+0000","last_clean":"2024-04-18T22:26:38.340902+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:38.340902+0000","last_undegraded":"2024-04-18T22:26:38.340902+0000","last_fullsized":"2024-04-18T22:26:38.340902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:52:17.813661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210997+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.210997+0000","last_peered":"2024-04-18T22:26:38.210997+0000","last_clean":"2024-04-18T22:26:38.210997+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.210997+0000","last_undegraded":"2024-04-18T22:26:38.210997+0000","last_fullsized":"2024-04-18T22:26:38.210997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:25.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339371+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339371+0000","last_peered":"2024-04-18T22:26:38.339371+0000","last_clean":"2024-04-18T22:26:38.339371+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339371+0000","last_undegraded":"2024-04-18T22:26:38.339371+0000","last_fullsized":"2024-04-18T22:26:38.339371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:42.038434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240563+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:38.240563+0000","last_peered":"2024-04-18T22:26:38.240563+0000","last_clean":"2024-04-18T22:26:38.240563+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:38.240563+0000","last_undegraded":"2024-04-18T22:26:38.240563+0000","last_fullsized":"2024-04-18T22:26:38.240563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:51:13.814270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240871+0000","last_change":"2024-04-18T22:26:37.248780+0000","last_active":"2024-04-18T22:26:38.240871+0000","last_peered":"2024-04-18T22:26:38.240871+0000","last_clean":"2024-04-18T22:26:38.240871+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:38.240871+0000","last_undegraded":"2024-04-18T22:26:38.240871+0000","last_fullsized":"2024-04-18T22:26:38.240871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210771+0000","last_change":"2024-04-18T22:26:26.049675+0000","last_active":"2024-04-18T22:26:38.210771+0000","last_peered":"2024-04-18T22:26:38.210771+0000","last_clean":"2024-04-18T22:26:38.210771+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:38.210771+0000","last_undegraded":"2024-04-18T22:26:38.210771+0000","last_fullsized":"2024-04-18T22:26:38.210771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:27.906774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338129+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:38.338129+0000","last_peered":"2024-04-18T22:26:38.338129+0000","last_clean":"2024-04-18T22:26:38.338129+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:38.338129+0000","last_undegraded":"2024-04-18T22:26:38.338129+0000","last_fullsized":"2024-04-18T22:26:38.338129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209335+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209335+0000","last_peered":"2024-04-18T22:26:38.209335+0000","last_clean":"2024-04-18T22:26:38.209335+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209335+0000","last_undegraded":"2024-04-18T22:26:38.209335+0000","last_fullsized":"2024-04-18T22:26:38.209335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:20:10.385885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210802+0000","last_change":"2024-04-18T22:26:26.053562+0000","last_active":"2024-04-18T22:26:38.210802+0000","last_peered":"2024-04-18T22:26:38.210802+0000","last_clean":"2024-04-18T22:26:38.210802+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:38.210802+0000","last_undegraded":"2024-04-18T22:26:38.210802+0000","last_fullsized":"2024-04-18T22:26:38.210802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:03.633713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340461+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:38.340461+0000","last_peered":"2024-04-18T22:26:38.340461+0000","last_clean":"2024-04-18T22:26:38.340461+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:38.340461+0000","last_undegraded":"2024-04-18T22:26:38.340461+0000","last_fullsized":"2024-04-18T22:26:38.340461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241369+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:38.241369+0000","last_peered":"2024-04-18T22:26:38.241369+0000","last_clean":"2024-04-18T22:26:38.241369+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:38.241369+0000","last_undegraded":"2024-04-18T22:26:38.241369+0000","last_fullsized":"2024-04-18T22:26:38.241369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:07.716048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211498+0000","last_change":"2024-04-18T22:26:26.057802+0000","last_active":"2024-04-18T22:26:38.211498+0000","last_peered":"2024-04-18T22:26:38.211498+0000","last_clean":"2024-04-18T22:26:38.211498+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:26:38.211498+0000","last_undegraded":"2024-04-18T22:26:38.211498+0000","last_fullsized":"2024-04-18T22:26:38.211498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:58:25.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339284+0000","last_change":"2024-04-18T22:26:34.217013+0000","last_active":"2024-04-18T22:26:38.339284+0000","last_peered":"2024-04-18T22:26:38.339284+0000","last_clean":"2024-04-18T22:26:38.339284+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:26:38.339284+0000","last_undegraded":"2024-04-18T22:26:38.339284+0000","last_fullsized":"2024-04-18T22:26:38.339284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:37.784925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210263+0000","last_change":"2024-04-18T22:26:26.050006+0000","last_active":"2024-04-18T22:26:38.210263+0000","last_peered":"2024-04-18T22:26:38.210263+0000","last_clean":"2024-04-18T22:26:38.210263+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:38.210263+0000","last_undegraded":"2024-04-18T22:26:38.210263+0000","last_fullsized":"2024-04-18T22:26:38.210263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:33:05.635452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338277+0000","last_change":"2024-04-18T22:26:26.060702+0000","last_active":"2024-04-18T22:26:38.338277+0000","last_peered":"2024-04-18T22:26:38.338277+0000","last_clean":"2024-04-18T22:26:38.338277+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:38.338277+0000","last_undegraded":"2024-04-18T22:26:38.338277+0000","last_fullsized":"2024-04-18T22:26:38.338277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:18:36.194967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239065+0000","last_change":"2024-04-18T22:26:37.250932+0000","last_active":"2024-04-18T22:26:38.239065+0000","last_peered":"2024-04-18T22:26:38.239065+0000","last_clean":"2024-04-18T22:26:38.239065+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:38.239065+0000","last_undegraded":"2024-04-18T22:26:38.239065+0000","last_fullsized":"2024-04-18T22:26:38.239065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:03.780060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210800+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210800+0000","last_peered":"2024-04-18T22:26:38.210800+0000","last_clean":"2024-04-18T22:26:38.210800+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210800+0000","last_undegraded":"2024-04-18T22:26:38.210800+0000","last_fullsized":"2024-04-18T22:26:38.210800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:29:28.068569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241235+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:38.241235+0000","last_peered":"2024-04-18T22:26:38.241235+0000","last_clean":"2024-04-18T22:26:38.241235+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:38.241235+0000","last_undegraded":"2024-04-18T22:26:38.241235+0000","last_fullsized":"2024-04-18T22:26:38.241235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:49:04.469351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211082+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211082+0000","last_peered":"2024-04-18T22:26:38.211082+0000","last_clean":"2024-04-18T22:26:38.211082+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211082+0000","last_undegraded":"2024-04-18T22:26:38.211082+0000","last_fullsized":"2024-04-18T22:26:38.211082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:49:20.072221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339128+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:26:38.339128+0000","last_peered":"2024-04-18T22:26:38.339128+0000","last_clean":"2024-04-18T22:26:38.339128+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:26:38.339128+0000","last_undegraded":"2024-04-18T22:26:38.339128+0000","last_fullsized":"2024-04-18T22:26:38.339128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340715+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340715+0000","last_peered":"2024-04-18T22:26:38.340715+0000","last_clean":"2024-04-18T22:26:38.340715+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340715+0000","last_undegraded":"2024-04-18T22:26:38.340715+0000","last_fullsized":"2024-04-18T22:26:38.340715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209937+0000","last_change":"2024-04-18T22:26:26.052492+0000","last_active":"2024-04-18T22:26:38.209937+0000","last_peered":"2024-04-18T22:26:38.209937+0000","last_clean":"2024-04-18T22:26:38.209937+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:38.209937+0000","last_undegraded":"2024-04-18T22:26:38.209937+0000","last_fullsized":"2024-04-18T22:26:38.209937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:39:51.990646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339820+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339820+0000","last_peered":"2024-04-18T22:26:38.339820+0000","last_clean":"2024-04-18T22:26:38.339820+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339820+0000","last_undegraded":"2024-04-18T22:26:38.339820+0000","last_fullsized":"2024-04-18T22:26:38.339820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:37.891211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239177+0000","last_change":"2024-04-18T22:26:37.250746+0000","last_active":"2024-04-18T22:26:38.239177+0000","last_peered":"2024-04-18T22:26:38.239177+0000","last_clean":"2024-04-18T22:26:38.239177+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:38.239177+0000","last_undegraded":"2024-04-18T22:26:38.239177+0000","last_fullsized":"2024-04-18T22:26:38.239177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:58:41.981858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240749+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:38.240749+0000","last_peered":"2024-04-18T22:26:38.240749+0000","last_clean":"2024-04-18T22:26:38.240749+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:38.240749+0000","last_undegraded":"2024-04-18T22:26:38.240749+0000","last_fullsized":"2024-04-18T22:26:38.240749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209349+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209349+0000","last_peered":"2024-04-18T22:26:38.209349+0000","last_clean":"2024-04-18T22:26:38.209349+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209349+0000","last_undegraded":"2024-04-18T22:26:38.209349+0000","last_fullsized":"2024-04-18T22:26:38.209349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:25:14.095942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210807+0000","last_change":"2024-04-18T22:26:26.056290+0000","last_active":"2024-04-18T22:26:38.210807+0000","last_peered":"2024-04-18T22:26:38.210807+0000","last_clean":"2024-04-18T22:26:38.210807+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:38.210807+0000","last_undegraded":"2024-04-18T22:26:38.210807+0000","last_fullsized":"2024-04-18T22:26:38.210807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:33:17.740653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209496+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209496+0000","last_peered":"2024-04-18T22:26:38.209496+0000","last_clean":"2024-04-18T22:26:38.209496+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209496+0000","last_undegraded":"2024-04-18T22:26:38.209496+0000","last_fullsized":"2024-04-18T22:26:38.209496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:02:53.362524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241052+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:38.241052+0000","last_peered":"2024-04-18T22:26:38.241052+0000","last_clean":"2024-04-18T22:26:38.241052+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:38.241052+0000","last_undegraded":"2024-04-18T22:26:38.241052+0000","last_fullsized":"2024-04-18T22:26:38.241052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:26.383475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239565+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:38.239565+0000","last_peered":"2024-04-18T22:26:38.239565+0000","last_clean":"2024-04-18T22:26:38.239565+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:38.239565+0000","last_undegraded":"2024-04-18T22:26:38.239565+0000","last_fullsized":"2024-04-18T22:26:38.239565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:44:05.966703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241128+0000","last_change":"2024-04-18T22:26:37.248320+0000","last_active":"2024-04-18T22:26:38.241128+0000","last_peered":"2024-04-18T22:26:38.241128+0000","last_clean":"2024-04-18T22:26:38.241128+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:38.241128+0000","last_undegraded":"2024-04-18T22:26:38.241128+0000","last_fullsized":"2024-04-18T22:26:38.241128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:36.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339542+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:38.339542+0000","last_peered":"2024-04-18T22:26:38.339542+0000","last_clean":"2024-04-18T22:26:38.339542+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:38.339542+0000","last_undegraded":"2024-04-18T22:26:38.339542+0000","last_fullsized":"2024-04-18T22:26:38.339542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339629+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339629+0000","last_peered":"2024-04-18T22:26:38.339629+0000","last_clean":"2024-04-18T22:26:38.339629+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339629+0000","last_undegraded":"2024-04-18T22:26:38.339629+0000","last_fullsized":"2024-04-18T22:26:38.339629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:30.374649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240601+0000","last_change":"2024-04-18T22:26:37.253099+0000","last_active":"2024-04-18T22:26:38.240601+0000","last_peered":"2024-04-18T22:26:38.240601+0000","last_clean":"2024-04-18T22:26:38.240601+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:38.240601+0000","last_undegraded":"2024-04-18T22:26:38.240601+0000","last_fullsized":"2024-04-18T22:26:38.240601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:08.780794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240100+0000","last_change":"2024-04-18T22:26:37.242866+0000","last_active":"2024-04-18T22:26:38.240100+0000","last_peered":"2024-04-18T22:26:38.240100+0000","last_clean":"2024-04-18T22:26:38.240100+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:26:38.240100+0000","last_undegraded":"2024-04-18T22:26:38.240100+0000","last_fullsized":"2024-04-18T22:26:38.240100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:36.007938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240704+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:38.240704+0000","last_peered":"2024-04-18T22:26:38.240704+0000","last_clean":"2024-04-18T22:26:38.240704+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:38.240704+0000","last_undegraded":"2024-04-18T22:26:38.240704+0000","last_fullsized":"2024-04-18T22:26:38.240704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:44:13.894111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209504+0000","last_change":"2024-04-18T22:26:26.052580+0000","last_active":"2024-04-18T22:26:38.209504+0000","last_peered":"2024-04-18T22:26:38.209504+0000","last_clean":"2024-04-18T22:26:38.209504+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:38.209504+0000","last_undegraded":"2024-04-18T22:26:38.209504+0000","last_fullsized":"2024-04-18T22:26:38.209504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:22:38.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240069+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:38.240069+0000","last_peered":"2024-04-18T22:26:38.240069+0000","last_clean":"2024-04-18T22:26:38.240069+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:38.240069+0000","last_undegraded":"2024-04-18T22:26:38.240069+0000","last_fullsized":"2024-04-18T22:26:38.240069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209668+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209668+0000","last_peered":"2024-04-18T22:26:38.209668+0000","last_clean":"2024-04-18T22:26:38.209668+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209668+0000","last_undegraded":"2024-04-18T22:26:38.209668+0000","last_fullsized":"2024-04-18T22:26:38.209668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:48:30.030678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340512+0000","last_change":"2024-04-18T22:26:26.070592+0000","last_active":"2024-04-18T22:26:38.340512+0000","last_peered":"2024-04-18T22:26:38.340512+0000","last_clean":"2024-04-18T22:26:38.340512+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:38.340512+0000","last_undegraded":"2024-04-18T22:26:38.340512+0000","last_fullsized":"2024-04-18T22:26:38.340512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:10:37.345485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240135+0000","last_change":"2024-04-18T22:26:37.247379+0000","last_active":"2024-04-18T22:26:38.240135+0000","last_peered":"2024-04-18T22:26:38.240135+0000","last_clean":"2024-04-18T22:26:38.240135+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:26:38.240135+0000","last_undegraded":"2024-04-18T22:26:38.240135+0000","last_fullsized":"2024-04-18T22:26:38.240135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:31.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241409+0000","last_change":"2024-04-18T22:26:37.251309+0000","last_active":"2024-04-18T22:26:38.241409+0000","last_peered":"2024-04-18T22:26:38.241409+0000","last_clean":"2024-04-18T22:26:38.241409+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:26:38.241409+0000","last_undegraded":"2024-04-18T22:26:38.241409+0000","last_fullsized":"2024-04-18T22:26:38.241409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:55:54.687547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338304+0000","last_change":"2024-04-18T22:26:34.216163+0000","last_active":"2024-04-18T22:26:38.338304+0000","last_peered":"2024-04-18T22:26:38.338304+0000","last_clean":"2024-04-18T22:26:38.338304+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:38.338304+0000","last_undegraded":"2024-04-18T22:26:38.338304+0000","last_fullsized":"2024-04-18T22:26:38.338304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:07.705933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241207+0000","last_change":"2024-04-18T22:26:37.253789+0000","last_active":"2024-04-18T22:26:38.241207+0000","last_peered":"2024-04-18T22:26:38.241207+0000","last_clean":"2024-04-18T22:26:38.241207+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:38.241207+0000","last_undegraded":"2024-04-18T22:26:38.241207+0000","last_fullsized":"2024-04-18T22:26:38.241207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:56.318449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240074+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:38.240074+0000","last_peered":"2024-04-18T22:26:38.240074+0000","last_clean":"2024-04-18T22:26:38.240074+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:38.240074+0000","last_undegraded":"2024-04-18T22:26:38.240074+0000","last_fullsized":"2024-04-18T22:26:38.240074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:51:03.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338880+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:26:38.338880+0000","last_peered":"2024-04-18T22:26:38.338880+0000","last_clean":"2024-04-18T22:26:38.338880+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:26:38.338880+0000","last_undegraded":"2024-04-18T22:26:38.338880+0000","last_fullsized":"2024-04-18T22:26:38.338880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211260+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211260+0000","last_peered":"2024-04-18T22:26:38.211260+0000","last_clean":"2024-04-18T22:26:38.211260+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211260+0000","last_undegraded":"2024-04-18T22:26:38.211260+0000","last_fullsized":"2024-04-18T22:26:38.211260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:40.351753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339220+0000","last_change":"2024-04-18T22:26:26.067550+0000","last_active":"2024-04-18T22:26:38.339220+0000","last_peered":"2024-04-18T22:26:38.339220+0000","last_clean":"2024-04-18T22:26:38.339220+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:38.339220+0000","last_undegraded":"2024-04-18T22:26:38.339220+0000","last_fullsized":"2024-04-18T22:26:38.339220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:49:44.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210088+0000","last_change":"2024-04-18T22:26:26.058882+0000","last_active":"2024-04-18T22:26:38.210088+0000","last_peered":"2024-04-18T22:26:38.210088+0000","last_clean":"2024-04-18T22:26:38.210088+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:38.210088+0000","last_undegraded":"2024-04-18T22:26:38.210088+0000","last_fullsized":"2024-04-18T22:26:38.210088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:47:23.720729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340082+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340082+0000","last_peered":"2024-04-18T22:26:38.340082+0000","last_clean":"2024-04-18T22:26:38.340082+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340082+0000","last_undegraded":"2024-04-18T22:26:38.340082+0000","last_fullsized":"2024-04-18T22:26:38.340082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240361+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:38.240361+0000","last_peered":"2024-04-18T22:26:38.240361+0000","last_clean":"2024-04-18T22:26:38.240361+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:38.240361+0000","last_undegraded":"2024-04-18T22:26:38.240361+0000","last_fullsized":"2024-04-18T22:26:38.240361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:32.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241182+0000","last_change":"2024-04-18T22:26:37.247834+0000","last_active":"2024-04-18T22:26:38.241182+0000","last_peered":"2024-04-18T22:26:38.241182+0000","last_clean":"2024-04-18T22:26:38.241182+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:38.241182+0000","last_undegraded":"2024-04-18T22:26:38.241182+0000","last_fullsized":"2024-04-18T22:26:38.241182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:45:58.922778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339394+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339394+0000","last_peered":"2024-04-18T22:26:38.339394+0000","last_clean":"2024-04-18T22:26:38.339394+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339394+0000","last_undegraded":"2024-04-18T22:26:38.339394+0000","last_fullsized":"2024-04-18T22:26:38.339394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240376+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:38.240376+0000","last_peered":"2024-04-18T22:26:38.240376+0000","last_clean":"2024-04-18T22:26:38.240376+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:38.240376+0000","last_undegraded":"2024-04-18T22:26:38.240376+0000","last_fullsized":"2024-04-18T22:26:38.240376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:12:44.602848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340352+0000","last_change":"2024-04-18T22:26:26.068751+0000","last_active":"2024-04-18T22:26:38.340352+0000","last_peered":"2024-04-18T22:26:38.340352+0000","last_clean":"2024-04-18T22:26:38.340352+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:38.340352+0000","last_undegraded":"2024-04-18T22:26:38.340352+0000","last_fullsized":"2024-04-18T22:26:38.340352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:58.461110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210605+0000","last_change":"2024-04-18T22:26:26.050076+0000","last_active":"2024-04-18T22:26:38.210605+0000","last_peered":"2024-04-18T22:26:38.210605+0000","last_clean":"2024-04-18T22:26:38.210605+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:38.210605+0000","last_undegraded":"2024-04-18T22:26:38.210605+0000","last_fullsized":"2024-04-18T22:26:38.210605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:43:48.685468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339247+0000","last_change":"2024-04-18T22:26:34.214565+0000","last_active":"2024-04-18T22:26:38.339247+0000","last_peered":"2024-04-18T22:26:38.339247+0000","last_clean":"2024-04-18T22:26:38.339247+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:38.339247+0000","last_undegraded":"2024-04-18T22:26:38.339247+0000","last_fullsized":"2024-04-18T22:26:38.339247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:00:17.620775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211798+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211798+0000","last_peered":"2024-04-18T22:26:38.211798+0000","last_clean":"2024-04-18T22:26:38.211798+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211798+0000","last_undegraded":"2024-04-18T22:26:38.211798+0000","last_fullsized":"2024-04-18T22:26:38.211798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:41.505834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209934+0000","last_change":"2024-04-18T22:26:26.049861+0000","last_active":"2024-04-18T22:26:38.209934+0000","last_peered":"2024-04-18T22:26:38.209934+0000","last_clean":"2024-04-18T22:26:38.209934+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:38.209934+0000","last_undegraded":"2024-04-18T22:26:38.209934+0000","last_fullsized":"2024-04-18T22:26:38.209934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:36:09.243064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340992+0000","last_change":"2024-04-18T22:26:26.071169+0000","last_active":"2024-04-18T22:26:38.340992+0000","last_peered":"2024-04-18T22:26:38.340992+0000","last_clean":"2024-04-18T22:26:38.340992+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:38.340992+0000","last_undegraded":"2024-04-18T22:26:38.340992+0000","last_fullsized":"2024-04-18T22:26:38.340992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:28.018224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339892+0000","last_change":"2024-04-18T22:26:34.216022+0000","last_active":"2024-04-18T22:26:38.339892+0000","last_peered":"2024-04-18T22:26:38.339892+0000","last_clean":"2024-04-18T22:26:38.339892+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:38.339892+0000","last_undegraded":"2024-04-18T22:26:38.339892+0000","last_fullsized":"2024-04-18T22:26:38.339892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:11:39.518803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338036+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338036+0000","last_peered":"2024-04-18T22:26:38.338036+0000","last_clean":"2024-04-18T22:26:38.338036+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338036+0000","last_undegraded":"2024-04-18T22:26:38.338036+0000","last_fullsized":"2024-04-18T22:26:38.338036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:37.018548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340605+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340605+0000","last_peered":"2024-04-18T22:26:38.340605+0000","last_clean":"2024-04-18T22:26:38.340605+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340605+0000","last_undegraded":"2024-04-18T22:26:38.340605+0000","last_fullsized":"2024-04-18T22:26:38.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:31.938261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240827+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:38.240827+0000","last_peered":"2024-04-18T22:26:38.240827+0000","last_clean":"2024-04-18T22:26:38.240827+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:38.240827+0000","last_undegraded":"2024-04-18T22:26:38.240827+0000","last_fullsized":"2024-04-18T22:26:38.240827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:39.646164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210238+0000","last_change":"2024-04-18T22:26:26.049399+0000","last_active":"2024-04-18T22:26:38.210238+0000","last_peered":"2024-04-18T22:26:38.210238+0000","last_clean":"2024-04-18T22:26:38.210238+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:26:38.210238+0000","last_undegraded":"2024-04-18T22:26:38.210238+0000","last_fullsized":"2024-04-18T22:26:38.210238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:58:33.909228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240079+0000","last_change":"2024-04-18T22:26:37.253737+0000","last_active":"2024-04-18T22:26:38.240079+0000","last_peered":"2024-04-18T22:26:38.240079+0000","last_clean":"2024-04-18T22:26:38.240079+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:38.240079+0000","last_undegraded":"2024-04-18T22:26:38.240079+0000","last_fullsized":"2024-04-18T22:26:38.240079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:58:47.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240898+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.240898+0000","last_peered":"2024-04-18T22:26:38.240898+0000","last_clean":"2024-04-18T22:26:38.240898+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.240898+0000","last_undegraded":"2024-04-18T22:26:38.240898+0000","last_fullsized":"2024-04-18T22:26:38.240898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:33.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338838+0000","last_change":"2024-04-18T22:26:26.059340+0000","last_active":"2024-04-18T22:26:38.338838+0000","last_peered":"2024-04-18T22:26:38.338838+0000","last_clean":"2024-04-18T22:26:38.338838+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:26:38.338838+0000","last_undegraded":"2024-04-18T22:26:38.338838+0000","last_fullsized":"2024-04-18T22:26:38.338838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:53:02.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211394+0000","last_change":"2024-04-18T22:26:26.058539+0000","last_active":"2024-04-18T22:26:38.211394+0000","last_peered":"2024-04-18T22:26:38.211394+0000","last_clean":"2024-04-18T22:26:38.211394+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:38.211394+0000","last_undegraded":"2024-04-18T22:26:38.211394+0000","last_fullsized":"2024-04-18T22:26:38.211394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:29.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240762+0000","last_change":"2024-04-18T22:26:37.245223+0000","last_active":"2024-04-18T22:26:38.240762+0000","last_peered":"2024-04-18T22:26:38.240762+0000","last_clean":"2024-04-18T22:26:38.240762+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:26:38.240762+0000","last_undegraded":"2024-04-18T22:26:38.240762+0000","last_fullsized":"2024-04-18T22:26:38.240762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:06:32.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338415+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338415+0000","last_peered":"2024-04-18T22:26:38.338415+0000","last_clean":"2024-04-18T22:26:38.338415+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338415+0000","last_undegraded":"2024-04-18T22:26:38.338415+0000","last_fullsized":"2024-04-18T22:26:38.338415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340762+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340762+0000","last_peered":"2024-04-18T22:26:38.340762+0000","last_clean":"2024-04-18T22:26:38.340762+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340762+0000","last_undegraded":"2024-04-18T22:26:38.340762+0000","last_fullsized":"2024-04-18T22:26:38.340762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338370+0000","last_change":"2024-04-18T22:26:34.221491+0000","last_active":"2024-04-18T22:26:38.338370+0000","last_peered":"2024-04-18T22:26:38.338370+0000","last_clean":"2024-04-18T22:26:38.338370+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:38.338370+0000","last_undegraded":"2024-04-18T22:26:38.338370+0000","last_fullsized":"2024-04-18T22:26:38.338370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211095+0000","last_change":"2024-04-18T22:26:26.053516+0000","last_active":"2024-04-18T22:26:38.211095+0000","last_peered":"2024-04-18T22:26:38.211095+0000","last_clean":"2024-04-18T22:26:38.211095+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:38.211095+0000","last_undegraded":"2024-04-18T22:26:38.211095+0000","last_fullsized":"2024-04-18T22:26:38.211095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:27:51.463983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338110+0000","last_change":"2024-04-18T22:26:34.215787+0000","last_active":"2024-04-18T22:26:38.338110+0000","last_peered":"2024-04-18T22:26:38.338110+0000","last_clean":"2024-04-18T22:26:38.338110+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:38.338110+0000","last_undegraded":"2024-04-18T22:26:38.338110+0000","last_fullsized":"2024-04-18T22:26:38.338110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:32.105080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340172+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:26:38.340172+0000","last_peered":"2024-04-18T22:26:38.340172+0000","last_clean":"2024-04-18T22:26:38.340172+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:26:38.340172+0000","last_undegraded":"2024-04-18T22:26:38.340172+0000","last_fullsized":"2024-04-18T22:26:38.340172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338576+0000","last_change":"2024-04-18T22:26:34.218064+0000","last_active":"2024-04-18T22:26:38.338576+0000","last_peered":"2024-04-18T22:26:38.338576+0000","last_clean":"2024-04-18T22:26:38.338576+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:26:38.338576+0000","last_undegraded":"2024-04-18T22:26:38.338576+0000","last_fullsized":"2024-04-18T22:26:38.338576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211199+0000","last_change":"2024-04-18T22:26:26.058084+0000","last_active":"2024-04-18T22:26:38.211199+0000","last_peered":"2024-04-18T22:26:38.211199+0000","last_clean":"2024-04-18T22:26:38.211199+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:26:38.211199+0000","last_undegraded":"2024-04-18T22:26:38.211199+0000","last_fullsized":"2024-04-18T22:26:38.211199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:53:45.480811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209942+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.209942+0000","last_peered":"2024-04-18T22:26:38.209942+0000","last_clean":"2024-04-18T22:26:38.209942+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.209942+0000","last_undegraded":"2024-04-18T22:26:38.209942+0000","last_fullsized":"2024-04-18T22:26:38.209942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:06:09.947309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211063+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211063+0000","last_peered":"2024-04-18T22:26:38.211063+0000","last_clean":"2024-04-18T22:26:38.211063+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211063+0000","last_undegraded":"2024-04-18T22:26:38.211063+0000","last_fullsized":"2024-04-18T22:26:38.211063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:00.276319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209992+0000","last_change":"2024-04-18T22:26:26.049503+0000","last_active":"2024-04-18T22:26:38.209992+0000","last_peered":"2024-04-18T22:26:38.209992+0000","last_clean":"2024-04-18T22:26:38.209992+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:38.209992+0000","last_undegraded":"2024-04-18T22:26:38.209992+0000","last_fullsized":"2024-04-18T22:26:38.209992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:47:34.371634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239668+0000","last_change":"2024-04-18T22:26:37.254175+0000","last_active":"2024-04-18T22:26:38.239668+0000","last_peered":"2024-04-18T22:26:38.239668+0000","last_clean":"2024-04-18T22:26:38.239668+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:38.239668+0000","last_undegraded":"2024-04-18T22:26:38.239668+0000","last_fullsized":"2024-04-18T22:26:38.239668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:48:00.575406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240707+0000","last_change":"2024-04-18T22:26:37.253978+0000","last_active":"2024-04-18T22:26:38.240707+0000","last_peered":"2024-04-18T22:26:38.240707+0000","last_clean":"2024-04-18T22:26:38.240707+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:38.240707+0000","last_undegraded":"2024-04-18T22:26:38.240707+0000","last_fullsized":"2024-04-18T22:26:38.240707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:38.579496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241140+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:38.241140+0000","last_peered":"2024-04-18T22:26:38.241140+0000","last_clean":"2024-04-18T22:26:38.241140+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:38.241140+0000","last_undegraded":"2024-04-18T22:26:38.241140+0000","last_fullsized":"2024-04-18T22:26:38.241140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:40:37.721198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210913+0000","last_change":"2024-04-18T22:26:26.054055+0000","last_active":"2024-04-18T22:26:38.210913+0000","last_peered":"2024-04-18T22:26:38.210913+0000","last_clean":"2024-04-18T22:26:38.210913+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:38.210913+0000","last_undegraded":"2024-04-18T22:26:38.210913+0000","last_fullsized":"2024-04-18T22:26:38.210913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:13.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209807+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209807+0000","last_peered":"2024-04-18T22:26:38.209807+0000","last_clean":"2024-04-18T22:26:38.209807+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209807+0000","last_undegraded":"2024-04-18T22:26:38.209807+0000","last_fullsized":"2024-04-18T22:26:38.209807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:26.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340548+0000","last_change":"2024-04-18T22:26:26.069294+0000","last_active":"2024-04-18T22:26:38.340548+0000","last_peered":"2024-04-18T22:26:38.340548+0000","last_clean":"2024-04-18T22:26:38.340548+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:38.340548+0000","last_undegraded":"2024-04-18T22:26:38.340548+0000","last_fullsized":"2024-04-18T22:26:38.340548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:46.805559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239221+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239221+0000","last_peered":"2024-04-18T22:26:38.239221+0000","last_clean":"2024-04-18T22:26:38.239221+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239221+0000","last_undegraded":"2024-04-18T22:26:38.239221+0000","last_fullsized":"2024-04-18T22:26:38.239221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339975+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.339975+0000","last_peered":"2024-04-18T22:26:38.339975+0000","last_clean":"2024-04-18T22:26:38.339975+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.339975+0000","last_undegraded":"2024-04-18T22:26:38.339975+0000","last_fullsized":"2024-04-18T22:26:38.339975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:02.769450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738425,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078668,"statfs":{"total":96636764160,"available":96336556032,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547205,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334752,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037088,"statfs":{"total":96636764160,"available":96293978112,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:26:43.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:26:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:43.849 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:26:43.866 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-04-18T22:26:42.949070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669824,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445696,"statfs":{"total":289910292480,"available":289224392704,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.732639"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211682+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211682+0000","last_peered":"2024-04-18T22:26:38.211682+0000","last_clean":"2024-04-18T22:26:38.211682+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211682+0000","last_undegraded":"2024-04-18T22:26:38.211682+0000","last_fullsized":"2024-04-18T22:26:38.211682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:16:02.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211107+0000","last_change":"2024-04-18T22:26:26.036531+0000","last_active":"2024-04-18T22:26:38.211107+0000","last_peered":"2024-04-18T22:26:38.211107+0000","last_clean":"2024-04-18T22:26:38.211107+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:26:38.211107+0000","last_undegraded":"2024-04-18T22:26:38.211107+0000","last_fullsized":"2024-04-18T22:26:38.211107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:17:26.168838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241489+0000","last_change":"2024-04-18T22:26:37.250608+0000","last_active":"2024-04-18T22:26:38.241489+0000","last_peered":"2024-04-18T22:26:38.241489+0000","last_clean":"2024-04-18T22:26:38.241489+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:38.241489+0000","last_undegraded":"2024-04-18T22:26:38.241489+0000","last_fullsized":"2024-04-18T22:26:38.241489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:32.948871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240445+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:38.240445+0000","last_peered":"2024-04-18T22:26:38.240445+0000","last_clean":"2024-04-18T22:26:38.240445+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:38.240445+0000","last_undegraded":"2024-04-18T22:26:38.240445+0000","last_fullsized":"2024-04-18T22:26:38.240445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:26:07.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210220+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210220+0000","last_peered":"2024-04-18T22:26:38.210220+0000","last_clean":"2024-04-18T22:26:38.210220+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210220+0000","last_undegraded":"2024-04-18T22:26:38.210220+0000","last_fullsized":"2024-04-18T22:26:38.210220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:35:56.119968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239263+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239263+0000","last_peered":"2024-04-18T22:26:38.239263+0000","last_clean":"2024-04-18T22:26:38.239263+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239263+0000","last_undegraded":"2024-04-18T22:26:38.239263+0000","last_fullsized":"2024-04-18T22:26:38.239263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:04:33.941262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239350+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:38.239350+0000","last_peered":"2024-04-18T22:26:38.239350+0000","last_clean":"2024-04-18T22:26:38.239350+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:38.239350+0000","last_undegraded":"2024-04-18T22:26:38.239350+0000","last_fullsized":"2024-04-18T22:26:38.239350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:21:05.317874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238671+0000","last_change":"2024-04-18T22:26:37.251857+0000","last_active":"2024-04-18T22:26:38.238671+0000","last_peered":"2024-04-18T22:26:38.238671+0000","last_clean":"2024-04-18T22:26:38.238671+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:26:38.238671+0000","last_undegraded":"2024-04-18T22:26:38.238671+0000","last_fullsized":"2024-04-18T22:26:38.238671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:07.544858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340474+0000","last_change":"2024-04-18T22:26:26.069968+0000","last_active":"2024-04-18T22:26:38.340474+0000","last_peered":"2024-04-18T22:26:38.340474+0000","last_clean":"2024-04-18T22:26:38.340474+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:38.340474+0000","last_undegraded":"2024-04-18T22:26:38.340474+0000","last_fullsized":"2024-04-18T22:26:38.340474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:16:05.689716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211534+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211534+0000","last_peered":"2024-04-18T22:26:38.211534+0000","last_clean":"2024-04-18T22:26:38.211534+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211534+0000","last_undegraded":"2024-04-18T22:26:38.211534+0000","last_fullsized":"2024-04-18T22:26:38.211534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:43:12.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241442+0000","last_change":"2024-04-18T22:26:37.250594+0000","last_active":"2024-04-18T22:26:38.241442+0000","last_peered":"2024-04-18T22:26:38.241442+0000","last_clean":"2024-04-18T22:26:38.241442+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:38.241442+0000","last_undegraded":"2024-04-18T22:26:38.241442+0000","last_fullsized":"2024-04-18T22:26:38.241442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:31.385259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211588+0000","last_change":"2024-04-18T22:26:26.057510+0000","last_active":"2024-04-18T22:26:38.211588+0000","last_peered":"2024-04-18T22:26:38.211588+0000","last_clean":"2024-04-18T22:26:38.211588+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:38.211588+0000","last_undegraded":"2024-04-18T22:26:38.211588+0000","last_fullsized":"2024-04-18T22:26:38.211588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:10.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210691+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.210691+0000","last_peered":"2024-04-18T22:26:38.210691+0000","last_clean":"2024-04-18T22:26:38.210691+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.210691+0000","last_undegraded":"2024-04-18T22:26:38.210691+0000","last_fullsized":"2024-04-18T22:26:38.210691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:45:48.363681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239408+0000","last_change":"2024-04-18T22:26:37.253632+0000","last_active":"2024-04-18T22:26:38.239408+0000","last_peered":"2024-04-18T22:26:38.239408+0000","last_clean":"2024-04-18T22:26:38.239408+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:38.239408+0000","last_undegraded":"2024-04-18T22:26:38.239408+0000","last_fullsized":"2024-04-18T22:26:38.239408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:28:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239299+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239299+0000","last_peered":"2024-04-18T22:26:38.239299+0000","last_clean":"2024-04-18T22:26:38.239299+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239299+0000","last_undegraded":"2024-04-18T22:26:38.239299+0000","last_fullsized":"2024-04-18T22:26:38.239299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:08.716543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238847+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:38.238847+0000","last_peered":"2024-04-18T22:26:38.238847+0000","last_clean":"2024-04-18T22:26:38.238847+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:38.238847+0000","last_undegraded":"2024-04-18T22:26:38.238847+0000","last_fullsized":"2024-04-18T22:26:38.238847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:50.499991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340902+0000","last_change":"2024-04-18T22:26:26.070529+0000","last_active":"2024-04-18T22:26:38.340902+0000","last_peered":"2024-04-18T22:26:38.340902+0000","last_clean":"2024-04-18T22:26:38.340902+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:38.340902+0000","last_undegraded":"2024-04-18T22:26:38.340902+0000","last_fullsized":"2024-04-18T22:26:38.340902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:52:17.813661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210997+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.210997+0000","last_peered":"2024-04-18T22:26:38.210997+0000","last_clean":"2024-04-18T22:26:38.210997+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.210997+0000","last_undegraded":"2024-04-18T22:26:38.210997+0000","last_fullsized":"2024-04-18T22:26:38.210997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:25.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339371+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339371+0000","last_peered":"2024-04-18T22:26:38.339371+0000","last_clean":"2024-04-18T22:26:38.339371+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339371+0000","last_undegraded":"2024-04-18T22:26:38.339371+0000","last_fullsized":"2024-04-18T22:26:38.339371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:42.038434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240563+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:38.240563+0000","last_peered":"2024-04-18T22:26:38.240563+0000","last_clean":"2024-04-18T22:26:38.240563+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:38.240563+0000","last_undegraded":"2024-04-18T22:26:38.240563+0000","last_fullsized":"2024-04-18T22:26:38.240563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:51:13.814270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240871+0000","last_change":"2024-04-18T22:26:37.248780+0000","last_active":"2024-04-18T22:26:38.240871+0000","last_peered":"2024-04-18T22:26:38.240871+0000","last_clean":"2024-04-18T22:26:38.240871+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:38.240871+0000","last_undegraded":"2024-04-18T22:26:38.240871+0000","last_fullsized":"2024-04-18T22:26:38.240871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210771+0000","last_change":"2024-04-18T22:26:26.049675+0000","last_active":"2024-04-18T22:26:38.210771+0000","last_peered":"2024-04-18T22:26:38.210771+0000","last_clean":"2024-04-18T22:26:38.210771+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:38.210771+0000","last_undegraded":"2024-04-18T22:26:38.210771+0000","last_fullsized":"2024-04-18T22:26:38.210771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:27.906774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338129+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:38.338129+0000","last_peered":"2024-04-18T22:26:38.338129+0000","last_clean":"2024-04-18T22:26:38.338129+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:38.338129+0000","last_undegraded":"2024-04-18T22:26:38.338129+0000","last_fullsized":"2024-04-18T22:26:38.338129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209335+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209335+0000","last_peered":"2024-04-18T22:26:38.209335+0000","last_clean":"2024-04-18T22:26:38.209335+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209335+0000","last_undegraded":"2024-04-18T22:26:38.209335+0000","last_fullsized":"2024-04-18T22:26:38.209335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:20:10.385885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210802+0000","last_change":"2024-04-18T22:26:26.053562+0000","last_active":"2024-04-18T22:26:38.210802+0000","last_peered":"2024-04-18T22:26:38.210802+0000","last_clean":"2024-04-18T22:26:38.210802+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:38.210802+0000","last_undegraded":"2024-04-18T22:26:38.210802+0000","last_fullsized":"2024-04-18T22:26:38.210802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:03.633713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340461+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:38.340461+0000","last_peered":"2024-04-18T22:26:38.340461+0000","last_clean":"2024-04-18T22:26:38.340461+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:38.340461+0000","last_undegraded":"2024-04-18T22:26:38.340461+0000","last_fullsized":"2024-04-18T22:26:38.340461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241369+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:38.241369+0000","last_peered":"2024-04-18T22:26:38.241369+0000","last_clean":"2024-04-18T22:26:38.241369+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:38.241369+0000","last_undegraded":"2024-04-18T22:26:38.241369+0000","last_fullsized":"2024-04-18T22:26:38.241369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:07.716048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211498+0000","last_change":"2024-04-18T22:26:26.057802+0000","last_active":"2024-04-18T22:26:38.211498+0000","last_peered":"2024-04-18T22:26:38.211498+0000","last_clean":"2024-04-18T22:26:38.211498+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:26:38.211498+0000","last_undegraded":"2024-04-18T22:26:38.211498+0000","last_fullsized":"2024-04-18T22:26:38.211498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:58:25.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339284+0000","last_change":"2024-04-18T22:26:34.217013+0000","last_active":"2024-04-18T22:26:38.339284+0000","last_peered":"2024-04-18T22:26:38.339284+0000","last_clean":"2024-04-18T22:26:38.339284+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:26:38.339284+0000","last_undegraded":"2024-04-18T22:26:38.339284+0000","last_fullsized":"2024-04-18T22:26:38.339284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:37.784925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210263+0000","last_change":"2024-04-18T22:26:26.050006+0000","last_active":"2024-04-18T22:26:38.210263+0000","last_peered":"2024-04-18T22:26:38.210263+0000","last_clean":"2024-04-18T22:26:38.210263+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:38.210263+0000","last_undegraded":"2024-04-18T22:26:38.210263+0000","last_fullsized":"2024-04-18T22:26:38.210263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:33:05.635452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338277+0000","last_change":"2024-04-18T22:26:26.060702+0000","last_active":"2024-04-18T22:26:38.338277+0000","last_peered":"2024-04-18T22:26:38.338277+0000","last_clean":"2024-04-18T22:26:38.338277+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:38.338277+0000","last_undegraded":"2024-04-18T22:26:38.338277+0000","last_fullsized":"2024-04-18T22:26:38.338277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:18:36.194967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239065+0000","last_change":"2024-04-18T22:26:37.250932+0000","last_active":"2024-04-18T22:26:38.239065+0000","last_peered":"2024-04-18T22:26:38.239065+0000","last_clean":"2024-04-18T22:26:38.239065+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:38.239065+0000","last_undegraded":"2024-04-18T22:26:38.239065+0000","last_fullsized":"2024-04-18T22:26:38.239065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:03.780060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210800+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210800+0000","last_peered":"2024-04-18T22:26:38.210800+0000","last_clean":"2024-04-18T22:26:38.210800+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210800+0000","last_undegraded":"2024-04-18T22:26:38.210800+0000","last_fullsized":"2024-04-18T22:26:38.210800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:29:28.068569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241235+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:38.241235+0000","last_peered":"2024-04-18T22:26:38.241235+0000","last_clean":"2024-04-18T22:26:38.241235+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:38.241235+0000","last_undegraded":"2024-04-18T22:26:38.241235+0000","last_fullsized":"2024-04-18T22:26:38.241235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:49:04.469351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211082+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211082+0000","last_peered":"2024-04-18T22:26:38.211082+0000","last_clean":"2024-04-18T22:26:38.211082+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211082+0000","last_undegraded":"2024-04-18T22:26:38.211082+0000","last_fullsized":"2024-04-18T22:26:38.211082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:49:20.072221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339128+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:26:38.339128+0000","last_peered":"2024-04-18T22:26:38.339128+0000","last_clean":"2024-04-18T22:26:38.339128+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:26:38.339128+0000","last_undegraded":"2024-04-18T22:26:38.339128+0000","last_fullsized":"2024-04-18T22:26:38.339128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340715+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340715+0000","last_peered":"2024-04-18T22:26:38.340715+0000","last_clean":"2024-04-18T22:26:38.340715+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340715+0000","last_undegraded":"2024-04-18T22:26:38.340715+0000","last_fullsized":"2024-04-18T22:26:38.340715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209937+0000","last_change":"2024-04-18T22:26:26.052492+0000","last_active":"2024-04-18T22:26:38.209937+0000","last_peered":"2024-04-18T22:26:38.209937+0000","last_clean":"2024-04-18T22:26:38.209937+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:38.209937+0000","last_undegraded":"2024-04-18T22:26:38.209937+0000","last_fullsized":"2024-04-18T22:26:38.209937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:39:51.990646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339820+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339820+0000","last_peered":"2024-04-18T22:26:38.339820+0000","last_clean":"2024-04-18T22:26:38.339820+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339820+0000","last_undegraded":"2024-04-18T22:26:38.339820+0000","last_fullsized":"2024-04-18T22:26:38.339820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:37.891211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239177+0000","last_change":"2024-04-18T22:26:37.250746+0000","last_active":"2024-04-18T22:26:38.239177+0000","last_peered":"2024-04-18T22:26:38.239177+0000","last_clean":"2024-04-18T22:26:38.239177+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:38.239177+0000","last_undegraded":"2024-04-18T22:26:38.239177+0000","last_fullsized":"2024-04-18T22:26:38.239177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:58:41.981858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240749+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:38.240749+0000","last_peered":"2024-04-18T22:26:38.240749+0000","last_clean":"2024-04-18T22:26:38.240749+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:38.240749+0000","last_undegraded":"2024-04-18T22:26:38.240749+0000","last_fullsized":"2024-04-18T22:26:38.240749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209349+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209349+0000","last_peered":"2024-04-18T22:26:38.209349+0000","last_clean":"2024-04-18T22:26:38.209349+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209349+0000","last_undegraded":"2024-04-18T22:26:38.209349+0000","last_fullsized":"2024-04-18T22:26:38.209349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:25:14.095942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210807+0000","last_change":"2024-04-18T22:26:26.056290+0000","last_active":"2024-04-18T22:26:38.210807+0000","last_peered":"2024-04-18T22:26:38.210807+0000","last_clean":"2024-04-18T22:26:38.210807+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:38.210807+0000","last_undegraded":"2024-04-18T22:26:38.210807+0000","last_fullsized":"2024-04-18T22:26:38.210807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:33:17.740653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209496+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209496+0000","last_peered":"2024-04-18T22:26:38.209496+0000","last_clean":"2024-04-18T22:26:38.209496+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209496+0000","last_undegraded":"2024-04-18T22:26:38.209496+0000","last_fullsized":"2024-04-18T22:26:38.209496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:02:53.362524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241052+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:38.241052+0000","last_peered":"2024-04-18T22:26:38.241052+0000","last_clean":"2024-04-18T22:26:38.241052+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:38.241052+0000","last_undegraded":"2024-04-18T22:26:38.241052+0000","last_fullsized":"2024-04-18T22:26:38.241052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:26.383475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239565+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:38.239565+0000","last_peered":"2024-04-18T22:26:38.239565+0000","last_clean":"2024-04-18T22:26:38.239565+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:38.239565+0000","last_undegraded":"2024-04-18T22:26:38.239565+0000","last_fullsized":"2024-04-18T22:26:38.239565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:44:05.966703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241128+0000","last_change":"2024-04-18T22:26:37.248320+0000","last_active":"2024-04-18T22:26:38.241128+0000","last_peered":"2024-04-18T22:26:38.241128+0000","last_clean":"2024-04-18T22:26:38.241128+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:38.241128+0000","last_undegraded":"2024-04-18T22:26:38.241128+0000","last_fullsized":"2024-04-18T22:26:38.241128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:36.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339542+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:38.339542+0000","last_peered":"2024-04-18T22:26:38.339542+0000","last_clean":"2024-04-18T22:26:38.339542+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:38.339542+0000","last_undegraded":"2024-04-18T22:26:38.339542+0000","last_fullsized":"2024-04-18T22:26:38.339542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339629+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339629+0000","last_peered":"2024-04-18T22:26:38.339629+0000","last_clean":"2024-04-18T22:26:38.339629+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339629+0000","last_undegraded":"2024-04-18T22:26:38.339629+0000","last_fullsized":"2024-04-18T22:26:38.339629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:30.374649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240601+0000","last_change":"2024-04-18T22:26:37.253099+0000","last_active":"2024-04-18T22:26:38.240601+0000","last_peered":"2024-04-18T22:26:38.240601+0000","last_clean":"2024-04-18T22:26:38.240601+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:38.240601+0000","last_undegraded":"2024-04-18T22:26:38.240601+0000","last_fullsized":"2024-04-18T22:26:38.240601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:08.780794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240100+0000","last_change":"2024-04-18T22:26:37.242866+0000","last_active":"2024-04-18T22:26:38.240100+0000","last_peered":"2024-04-18T22:26:38.240100+0000","last_clean":"2024-04-18T22:26:38.240100+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:26:38.240100+0000","last_undegraded":"2024-04-18T22:26:38.240100+0000","last_fullsized":"2024-04-18T22:26:38.240100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:36.007938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240704+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:38.240704+0000","last_peered":"2024-04-18T22:26:38.240704+0000","last_clean":"2024-04-18T22:26:38.240704+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:38.240704+0000","last_undegraded":"2024-04-18T22:26:38.240704+0000","last_fullsized":"2024-04-18T22:26:38.240704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:44:13.894111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209504+0000","last_change":"2024-04-18T22:26:26.052580+0000","last_active":"2024-04-18T22:26:38.209504+0000","last_peered":"2024-04-18T22:26:38.209504+0000","last_clean":"2024-04-18T22:26:38.209504+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:38.209504+0000","last_undegraded":"2024-04-18T22:26:38.209504+0000","last_fullsized":"2024-04-18T22:26:38.209504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:22:38.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240069+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:38.240069+0000","last_peered":"2024-04-18T22:26:38.240069+0000","last_clean":"2024-04-18T22:26:38.240069+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:38.240069+0000","last_undegraded":"2024-04-18T22:26:38.240069+0000","last_fullsized":"2024-04-18T22:26:38.240069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209668+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209668+0000","last_peered":"2024-04-18T22:26:38.209668+0000","last_clean":"2024-04-18T22:26:38.209668+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209668+0000","last_undegraded":"2024-04-18T22:26:38.209668+0000","last_fullsized":"2024-04-18T22:26:38.209668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:48:30.030678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340512+0000","last_change":"2024-04-18T22:26:26.070592+0000","last_active":"2024-04-18T22:26:38.340512+0000","last_peered":"2024-04-18T22:26:38.340512+0000","last_clean":"2024-04-18T22:26:38.340512+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:38.340512+0000","last_undegraded":"2024-04-18T22:26:38.340512+0000","last_fullsized":"2024-04-18T22:26:38.340512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:10:37.345485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240135+0000","last_change":"2024-04-18T22:26:37.247379+0000","last_active":"2024-04-18T22:26:38.240135+0000","last_peered":"2024-04-18T22:26:38.240135+0000","last_clean":"2024-04-18T22:26:38.240135+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:26:38.240135+0000","last_undegraded":"2024-04-18T22:26:38.240135+0000","last_fullsized":"2024-04-18T22:26:38.240135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:31.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241409+0000","last_change":"2024-04-18T22:26:37.251309+0000","last_active":"2024-04-18T22:26:38.241409+0000","last_peered":"2024-04-18T22:26:38.241409+0000","last_clean":"2024-04-18T22:26:38.241409+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:26:38.241409+0000","last_undegraded":"2024-04-18T22:26:38.241409+0000","last_fullsized":"2024-04-18T22:26:38.241409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:55:54.687547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338304+0000","last_change":"2024-04-18T22:26:34.216163+0000","last_active":"2024-04-18T22:26:38.338304+0000","last_peered":"2024-04-18T22:26:38.338304+0000","last_clean":"2024-04-18T22:26:38.338304+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:38.338304+0000","last_undegraded":"2024-04-18T22:26:38.338304+0000","last_fullsized":"2024-04-18T22:26:38.338304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:07.705933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241207+0000","last_change":"2024-04-18T22:26:37.253789+0000","last_active":"2024-04-18T22:26:38.241207+0000","last_peered":"2024-04-18T22:26:38.241207+0000","last_clean":"2024-04-18T22:26:38.241207+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:38.241207+0000","last_undegraded":"2024-04-18T22:26:38.241207+0000","last_fullsized":"2024-04-18T22:26:38.241207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:56.318449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240074+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:38.240074+0000","last_peered":"2024-04-18T22:26:38.240074+0000","last_clean":"2024-04-18T22:26:38.240074+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:38.240074+0000","last_undegraded":"2024-04-18T22:26:38.240074+0000","last_fullsized":"2024-04-18T22:26:38.240074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:51:03.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338880+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:26:38.338880+0000","last_peered":"2024-04-18T22:26:38.338880+0000","last_clean":"2024-04-18T22:26:38.338880+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:26:38.338880+0000","last_undegraded":"2024-04-18T22:26:38.338880+0000","last_fullsized":"2024-04-18T22:26:38.338880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211260+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211260+0000","last_peered":"2024-04-18T22:26:38.211260+0000","last_clean":"2024-04-18T22:26:38.211260+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211260+0000","last_undegraded":"2024-04-18T22:26:38.211260+0000","last_fullsized":"2024-04-18T22:26:38.211260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:40.351753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339220+0000","last_change":"2024-04-18T22:26:26.067550+0000","last_active":"2024-04-18T22:26:38.339220+0000","last_peered":"2024-04-18T22:26:38.339220+0000","last_clean":"2024-04-18T22:26:38.339220+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:38.339220+0000","last_undegraded":"2024-04-18T22:26:38.339220+0000","last_fullsized":"2024-04-18T22:26:38.339220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:49:44.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210088+0000","last_change":"2024-04-18T22:26:26.058882+0000","last_active":"2024-04-18T22:26:38.210088+0000","last_peered":"2024-04-18T22:26:38.210088+0000","last_clean":"2024-04-18T22:26:38.210088+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:38.210088+0000","last_undegraded":"2024-04-18T22:26:38.210088+0000","last_fullsized":"2024-04-18T22:26:38.210088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:47:23.720729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340082+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340082+0000","last_peered":"2024-04-18T22:26:38.340082+0000","last_clean":"2024-04-18T22:26:38.340082+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340082+0000","last_undegraded":"2024-04-18T22:26:38.340082+0000","last_fullsized":"2024-04-18T22:26:38.340082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240361+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:38.240361+0000","last_peered":"2024-04-18T22:26:38.240361+0000","last_clean":"2024-04-18T22:26:38.240361+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:38.240361+0000","last_undegraded":"2024-04-18T22:26:38.240361+0000","last_fullsized":"2024-04-18T22:26:38.240361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:32.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241182+0000","last_change":"2024-04-18T22:26:37.247834+0000","last_active":"2024-04-18T22:26:38.241182+0000","last_peered":"2024-04-18T22:26:38.241182+0000","last_clean":"2024-04-18T22:26:38.241182+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:38.241182+0000","last_undegraded":"2024-04-18T22:26:38.241182+0000","last_fullsized":"2024-04-18T22:26:38.241182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:45:58.922778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339394+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339394+0000","last_peered":"2024-04-18T22:26:38.339394+0000","last_clean":"2024-04-18T22:26:38.339394+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339394+0000","last_undegraded":"2024-04-18T22:26:38.339394+0000","last_fullsized":"2024-04-18T22:26:38.339394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240376+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:38.240376+0000","last_peered":"2024-04-18T22:26:38.240376+0000","last_clean":"2024-04-18T22:26:38.240376+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:38.240376+0000","last_undegraded":"2024-04-18T22:26:38.240376+0000","last_fullsized":"2024-04-18T22:26:38.240376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:12:44.602848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340352+0000","last_change":"2024-04-18T22:26:26.068751+0000","last_active":"2024-04-18T22:26:38.340352+0000","last_peered":"2024-04-18T22:26:38.340352+0000","last_clean":"2024-04-18T22:26:38.340352+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:38.340352+0000","last_undegraded":"2024-04-18T22:26:38.340352+0000","last_fullsized":"2024-04-18T22:26:38.340352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:58.461110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210605+0000","last_change":"2024-04-18T22:26:26.050076+0000","last_active":"2024-04-18T22:26:38.210605+0000","last_peered":"2024-04-18T22:26:38.210605+0000","last_clean":"2024-04-18T22:26:38.210605+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:38.210605+0000","last_undegraded":"2024-04-18T22:26:38.210605+0000","last_fullsized":"2024-04-18T22:26:38.210605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:43:48.685468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339247+0000","last_change":"2024-04-18T22:26:34.214565+0000","last_active":"2024-04-18T22:26:38.339247+0000","last_peered":"2024-04-18T22:26:38.339247+0000","last_clean":"2024-04-18T22:26:38.339247+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:38.339247+0000","last_undegraded":"2024-04-18T22:26:38.339247+0000","last_fullsized":"2024-04-18T22:26:38.339247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:00:17.620775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211798+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211798+0000","last_peered":"2024-04-18T22:26:38.211798+0000","last_clean":"2024-04-18T22:26:38.211798+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211798+0000","last_undegraded":"2024-04-18T22:26:38.211798+0000","last_fullsized":"2024-04-18T22:26:38.211798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:41.505834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209934+0000","last_change":"2024-04-18T22:26:26.049861+0000","last_active":"2024-04-18T22:26:38.209934+0000","last_peered":"2024-04-18T22:26:38.209934+0000","last_clean":"2024-04-18T22:26:38.209934+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:38.209934+0000","last_undegraded":"2024-04-18T22:26:38.209934+0000","last_fullsized":"2024-04-18T22:26:38.209934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:36:09.243064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340992+0000","last_change":"2024-04-18T22:26:26.071169+0000","last_active":"2024-04-18T22:26:38.340992+0000","last_peered":"2024-04-18T22:26:38.340992+0000","last_clean":"2024-04-18T22:26:38.340992+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:38.340992+0000","last_undegraded":"2024-04-18T22:26:38.340992+0000","last_fullsized":"2024-04-18T22:26:38.340992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:28.018224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339892+0000","last_change":"2024-04-18T22:26:34.216022+0000","last_active":"2024-04-18T22:26:38.339892+0000","last_peered":"2024-04-18T22:26:38.339892+0000","last_clean":"2024-04-18T22:26:38.339892+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:38.339892+0000","last_undegraded":"2024-04-18T22:26:38.339892+0000","last_fullsized":"2024-04-18T22:26:38.339892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:11:39.518803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338036+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338036+0000","last_peered":"2024-04-18T22:26:38.338036+0000","last_clean":"2024-04-18T22:26:38.338036+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338036+0000","last_undegraded":"2024-04-18T22:26:38.338036+0000","last_fullsized":"2024-04-18T22:26:38.338036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:37.018548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340605+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340605+0000","last_peered":"2024-04-18T22:26:38.340605+0000","last_clean":"2024-04-18T22:26:38.340605+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340605+0000","last_undegraded":"2024-04-18T22:26:38.340605+0000","last_fullsized":"2024-04-18T22:26:38.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:31.938261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240827+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:38.240827+0000","last_peered":"2024-04-18T22:26:38.240827+0000","last_clean":"2024-04-18T22:26:38.240827+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:38.240827+0000","last_undegraded":"2024-04-18T22:26:38.240827+0000","last_fullsized":"2024-04-18T22:26:38.240827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:39.646164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210238+0000","last_change":"2024-04-18T22:26:26.049399+0000","last_active":"2024-04-18T22:26:38.210238+0000","last_peered":"2024-04-18T22:26:38.210238+0000","last_clean":"2024-04-18T22:26:38.210238+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:26:38.210238+0000","last_undegraded":"2024-04-18T22:26:38.210238+0000","last_fullsized":"2024-04-18T22:26:38.210238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:58:33.909228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240079+0000","last_change":"2024-04-18T22:26:37.253737+0000","last_active":"2024-04-18T22:26:38.240079+0000","last_peered":"2024-04-18T22:26:38.240079+0000","last_clean":"2024-04-18T22:26:38.240079+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:38.240079+0000","last_undegraded":"2024-04-18T22:26:38.240079+0000","last_fullsized":"2024-04-18T22:26:38.240079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:58:47.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240898+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.240898+0000","last_peered":"2024-04-18T22:26:38.240898+0000","last_clean":"2024-04-18T22:26:38.240898+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.240898+0000","last_undegraded":"2024-04-18T22:26:38.240898+0000","last_fullsized":"2024-04-18T22:26:38.240898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:33.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338838+0000","last_change":"2024-04-18T22:26:26.059340+0000","last_active":"2024-04-18T22:26:38.338838+0000","last_peered":"2024-04-18T22:26:38.338838+0000","last_clean":"2024-04-18T22:26:38.338838+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:26:38.338838+0000","last_undegraded":"2024-04-18T22:26:38.338838+0000","last_fullsized":"2024-04-18T22:26:38.338838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:53:02.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211394+0000","last_change":"2024-04-18T22:26:26.058539+0000","last_active":"2024-04-18T22:26:38.211394+0000","last_peered":"2024-04-18T22:26:38.211394+0000","last_clean":"2024-04-18T22:26:38.211394+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:38.211394+0000","last_undegraded":"2024-04-18T22:26:38.211394+0000","last_fullsized":"2024-04-18T22:26:38.211394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:29.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240762+0000","last_change":"2024-04-18T22:26:37.245223+0000","last_active":"2024-04-18T22:26:38.240762+0000","last_peered":"2024-04-18T22:26:38.240762+0000","last_clean":"2024-04-18T22:26:38.240762+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:26:38.240762+0000","last_undegraded":"2024-04-18T22:26:38.240762+0000","last_fullsized":"2024-04-18T22:26:38.240762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:06:32.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338415+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338415+0000","last_peered":"2024-04-18T22:26:38.338415+0000","last_clean":"2024-04-18T22:26:38.338415+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338415+0000","last_undegraded":"2024-04-18T22:26:38.338415+0000","last_fullsized":"2024-04-18T22:26:38.338415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340762+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340762+0000","last_peered":"2024-04-18T22:26:38.340762+0000","last_clean":"2024-04-18T22:26:38.340762+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340762+0000","last_undegraded":"2024-04-18T22:26:38.340762+0000","last_fullsized":"2024-04-18T22:26:38.340762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338370+0000","last_change":"2024-04-18T22:26:34.221491+0000","last_active":"2024-04-18T22:26:38.338370+0000","last_peered":"2024-04-18T22:26:38.338370+0000","last_clean":"2024-04-18T22:26:38.338370+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:38.338370+0000","last_undegraded":"2024-04-18T22:26:38.338370+0000","last_fullsized":"2024-04-18T22:26:38.338370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211095+0000","last_change":"2024-04-18T22:26:26.053516+0000","last_active":"2024-04-18T22:26:38.211095+0000","last_peered":"2024-04-18T22:26:38.211095+0000","last_clean":"2024-04-18T22:26:38.211095+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:38.211095+0000","last_undegraded":"2024-04-18T22:26:38.211095+0000","last_fullsized":"2024-04-18T22:26:38.211095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:27:51.463983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338110+0000","last_change":"2024-04-18T22:26:34.215787+0000","last_active":"2024-04-18T22:26:38.338110+0000","last_peered":"2024-04-18T22:26:38.338110+0000","last_clean":"2024-04-18T22:26:38.338110+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:38.338110+0000","last_undegraded":"2024-04-18T22:26:38.338110+0000","last_fullsized":"2024-04-18T22:26:38.338110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:32.105080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340172+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:26:38.340172+0000","last_peered":"2024-04-18T22:26:38.340172+0000","last_clean":"2024-04-18T22:26:38.340172+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:26:38.340172+0000","last_undegraded":"2024-04-18T22:26:38.340172+0000","last_fullsized":"2024-04-18T22:26:38.340172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338576+0000","last_change":"2024-04-18T22:26:34.218064+0000","last_active":"2024-04-18T22:26:38.338576+0000","last_peered":"2024-04-18T22:26:38.338576+0000","last_clean":"2024-04-18T22:26:38.338576+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:26:38.338576+0000","last_undegraded":"2024-04-18T22:26:38.338576+0000","last_fullsized":"2024-04-18T22:26:38.338576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211199+0000","last_change":"2024-04-18T22:26:26.058084+0000","last_active":"2024-04-18T22:26:38.211199+0000","last_peered":"2024-04-18T22:26:38.211199+0000","last_clean":"2024-04-18T22:26:38.211199+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:26:38.211199+0000","last_undegraded":"2024-04-18T22:26:38.211199+0000","last_fullsized":"2024-04-18T22:26:38.211199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:53:45.480811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209942+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.209942+0000","last_peered":"2024-04-18T22:26:38.209942+0000","last_clean":"2024-04-18T22:26:38.209942+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.209942+0000","last_undegraded":"2024-04-18T22:26:38.209942+0000","last_fullsized":"2024-04-18T22:26:38.209942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:06:09.947309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211063+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211063+0000","last_peered":"2024-04-18T22:26:38.211063+0000","last_clean":"2024-04-18T22:26:38.211063+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211063+0000","last_undegraded":"2024-04-18T22:26:38.211063+0000","last_fullsized":"2024-04-18T22:26:38.211063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:00.276319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209992+0000","last_change":"2024-04-18T22:26:26.049503+0000","last_active":"2024-04-18T22:26:38.209992+0000","last_peered":"2024-04-18T22:26:38.209992+0000","last_clean":"2024-04-18T22:26:38.209992+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:38.209992+0000","last_undegraded":"2024-04-18T22:26:38.209992+0000","last_fullsized":"2024-04-18T22:26:38.209992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:47:34.371634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239668+0000","last_change":"2024-04-18T22:26:37.254175+0000","last_active":"2024-04-18T22:26:38.239668+0000","last_peered":"2024-04-18T22:26:38.239668+0000","last_clean":"2024-04-18T22:26:38.239668+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:38.239668+0000","last_undegraded":"2024-04-18T22:26:38.239668+0000","last_fullsized":"2024-04-18T22:26:38.239668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:48:00.575406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240707+0000","last_change":"2024-04-18T22:26:37.253978+0000","last_active":"2024-04-18T22:26:38.240707+0000","last_peered":"2024-04-18T22:26:38.240707+0000","last_clean":"2024-04-18T22:26:38.240707+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:38.240707+0000","last_undegraded":"2024-04-18T22:26:38.240707+0000","last_fullsized":"2024-04-18T22:26:38.240707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:38.579496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241140+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:38.241140+0000","last_peered":"2024-04-18T22:26:38.241140+0000","last_clean":"2024-04-18T22:26:38.241140+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:38.241140+0000","last_undegraded":"2024-04-18T22:26:38.241140+0000","last_fullsized":"2024-04-18T22:26:38.241140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:40:37.721198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210913+0000","last_change":"2024-04-18T22:26:26.054055+0000","last_active":"2024-04-18T22:26:38.210913+0000","last_peered":"2024-04-18T22:26:38.210913+0000","last_clean":"2024-04-18T22:26:38.210913+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:38.210913+0000","last_undegraded":"2024-04-18T22:26:38.210913+0000","last_fullsized":"2024-04-18T22:26:38.210913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:13.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209807+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209807+0000","last_peered":"2024-04-18T22:26:38.209807+0000","last_clean":"2024-04-18T22:26:38.209807+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209807+0000","last_undegraded":"2024-04-18T22:26:38.209807+0000","last_fullsized":"2024-04-18T22:26:38.209807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:26.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340548+0000","last_change":"2024-04-18T22:26:26.069294+0000","last_active":"2024-04-18T22:26:38.340548+0000","last_peered":"2024-04-18T22:26:38.340548+0000","last_clean":"2024-04-18T22:26:38.340548+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:38.340548+0000","last_undegraded":"2024-04-18T22:26:38.340548+0000","last_fullsized":"2024-04-18T22:26:38.340548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:46.805559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239221+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239221+0000","last_peered":"2024-04-18T22:26:38.239221+0000","last_clean":"2024-04-18T22:26:38.239221+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239221+0000","last_undegraded":"2024-04-18T22:26:38.239221+0000","last_fullsized":"2024-04-18T22:26:38.239221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339975+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.339975+0000","last_peered":"2024-04-18T22:26:38.339975+0000","last_clean":"2024-04-18T22:26:38.339975+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.339975+0000","last_undegraded":"2024-04-18T22:26:38.339975+0000","last_fullsized":"2024-04-18T22:26:38.339975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:02.769450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738425,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078668,"statfs":{"total":96636764160,"available":96336556032,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547205,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334752,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037088,"statfs":{"total":96636764160,"available":96293978112,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:26:43.871 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-18T22:26:43.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:26:44.216 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:26:44.233 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-04-18T22:26:42.949070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669824,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445696,"statfs":{"total":289910292480,"available":289224392704,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.732639"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211682+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211682+0000","last_peered":"2024-04-18T22:26:38.211682+0000","last_clean":"2024-04-18T22:26:38.211682+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211682+0000","last_undegraded":"2024-04-18T22:26:38.211682+0000","last_fullsized":"2024-04-18T22:26:38.211682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:16:02.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211107+0000","last_change":"2024-04-18T22:26:26.036531+0000","last_active":"2024-04-18T22:26:38.211107+0000","last_peered":"2024-04-18T22:26:38.211107+0000","last_clean":"2024-04-18T22:26:38.211107+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:26:38.211107+0000","last_undegraded":"2024-04-18T22:26:38.211107+0000","last_fullsized":"2024-04-18T22:26:38.211107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:17:26.168838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241489+0000","last_change":"2024-04-18T22:26:37.250608+0000","last_active":"2024-04-18T22:26:38.241489+0000","last_peered":"2024-04-18T22:26:38.241489+0000","last_clean":"2024-04-18T22:26:38.241489+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:38.241489+0000","last_undegraded":"2024-04-18T22:26:38.241489+0000","last_fullsized":"2024-04-18T22:26:38.241489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:32.948871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240445+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:38.240445+0000","last_peered":"2024-04-18T22:26:38.240445+0000","last_clean":"2024-04-18T22:26:38.240445+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:38.240445+0000","last_undegraded":"2024-04-18T22:26:38.240445+0000","last_fullsized":"2024-04-18T22:26:38.240445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:26:07.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210220+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210220+0000","last_peered":"2024-04-18T22:26:38.210220+0000","last_clean":"2024-04-18T22:26:38.210220+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210220+0000","last_undegraded":"2024-04-18T22:26:38.210220+0000","last_fullsized":"2024-04-18T22:26:38.210220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:35:56.119968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239263+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239263+0000","last_peered":"2024-04-18T22:26:38.239263+0000","last_clean":"2024-04-18T22:26:38.239263+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239263+0000","last_undegraded":"2024-04-18T22:26:38.239263+0000","last_fullsized":"2024-04-18T22:26:38.239263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:04:33.941262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239350+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:38.239350+0000","last_peered":"2024-04-18T22:26:38.239350+0000","last_clean":"2024-04-18T22:26:38.239350+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:38.239350+0000","last_undegraded":"2024-04-18T22:26:38.239350+0000","last_fullsized":"2024-04-18T22:26:38.239350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:21:05.317874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238671+0000","last_change":"2024-04-18T22:26:37.251857+0000","last_active":"2024-04-18T22:26:38.238671+0000","last_peered":"2024-04-18T22:26:38.238671+0000","last_clean":"2024-04-18T22:26:38.238671+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:26:38.238671+0000","last_undegraded":"2024-04-18T22:26:38.238671+0000","last_fullsized":"2024-04-18T22:26:38.238671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:07.544858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340474+0000","last_change":"2024-04-18T22:26:26.069968+0000","last_active":"2024-04-18T22:26:38.340474+0000","last_peered":"2024-04-18T22:26:38.340474+0000","last_clean":"2024-04-18T22:26:38.340474+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:38.340474+0000","last_undegraded":"2024-04-18T22:26:38.340474+0000","last_fullsized":"2024-04-18T22:26:38.340474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:16:05.689716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211534+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211534+0000","last_peered":"2024-04-18T22:26:38.211534+0000","last_clean":"2024-04-18T22:26:38.211534+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211534+0000","last_undegraded":"2024-04-18T22:26:38.211534+0000","last_fullsized":"2024-04-18T22:26:38.211534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:43:12.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241442+0000","last_change":"2024-04-18T22:26:37.250594+0000","last_active":"2024-04-18T22:26:38.241442+0000","last_peered":"2024-04-18T22:26:38.241442+0000","last_clean":"2024-04-18T22:26:38.241442+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:38.241442+0000","last_undegraded":"2024-04-18T22:26:38.241442+0000","last_fullsized":"2024-04-18T22:26:38.241442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:31.385259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211588+0000","last_change":"2024-04-18T22:26:26.057510+0000","last_active":"2024-04-18T22:26:38.211588+0000","last_peered":"2024-04-18T22:26:38.211588+0000","last_clean":"2024-04-18T22:26:38.211588+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:38.211588+0000","last_undegraded":"2024-04-18T22:26:38.211588+0000","last_fullsized":"2024-04-18T22:26:38.211588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:10.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210691+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.210691+0000","last_peered":"2024-04-18T22:26:38.210691+0000","last_clean":"2024-04-18T22:26:38.210691+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.210691+0000","last_undegraded":"2024-04-18T22:26:38.210691+0000","last_fullsized":"2024-04-18T22:26:38.210691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:45:48.363681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239408+0000","last_change":"2024-04-18T22:26:37.253632+0000","last_active":"2024-04-18T22:26:38.239408+0000","last_peered":"2024-04-18T22:26:38.239408+0000","last_clean":"2024-04-18T22:26:38.239408+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:38.239408+0000","last_undegraded":"2024-04-18T22:26:38.239408+0000","last_fullsized":"2024-04-18T22:26:38.239408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:28:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239299+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239299+0000","last_peered":"2024-04-18T22:26:38.239299+0000","last_clean":"2024-04-18T22:26:38.239299+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239299+0000","last_undegraded":"2024-04-18T22:26:38.239299+0000","last_fullsized":"2024-04-18T22:26:38.239299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:08.716543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238847+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:38.238847+0000","last_peered":"2024-04-18T22:26:38.238847+0000","last_clean":"2024-04-18T22:26:38.238847+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:38.238847+0000","last_undegraded":"2024-04-18T22:26:38.238847+0000","last_fullsized":"2024-04-18T22:26:38.238847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:50.499991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340902+0000","last_change":"2024-04-18T22:26:26.070529+0000","last_active":"2024-04-18T22:26:38.340902+0000","last_peered":"2024-04-18T22:26:38.340902+0000","last_clean":"2024-04-18T22:26:38.340902+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:38.340902+0000","last_undegraded":"2024-04-18T22:26:38.340902+0000","last_fullsized":"2024-04-18T22:26:38.340902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:52:17.813661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210997+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.210997+0000","last_peered":"2024-04-18T22:26:38.210997+0000","last_clean":"2024-04-18T22:26:38.210997+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.210997+0000","last_undegraded":"2024-04-18T22:26:38.210997+0000","last_fullsized":"2024-04-18T22:26:38.210997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:25.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339371+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339371+0000","last_peered":"2024-04-18T22:26:38.339371+0000","last_clean":"2024-04-18T22:26:38.339371+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339371+0000","last_undegraded":"2024-04-18T22:26:38.339371+0000","last_fullsized":"2024-04-18T22:26:38.339371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:42.038434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240563+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:38.240563+0000","last_peered":"2024-04-18T22:26:38.240563+0000","last_clean":"2024-04-18T22:26:38.240563+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:38.240563+0000","last_undegraded":"2024-04-18T22:26:38.240563+0000","last_fullsized":"2024-04-18T22:26:38.240563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:51:13.814270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240871+0000","last_change":"2024-04-18T22:26:37.248780+0000","last_active":"2024-04-18T22:26:38.240871+0000","last_peered":"2024-04-18T22:26:38.240871+0000","last_clean":"2024-04-18T22:26:38.240871+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:38.240871+0000","last_undegraded":"2024-04-18T22:26:38.240871+0000","last_fullsized":"2024-04-18T22:26:38.240871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210771+0000","last_change":"2024-04-18T22:26:26.049675+0000","last_active":"2024-04-18T22:26:38.210771+0000","last_peered":"2024-04-18T22:26:38.210771+0000","last_clean":"2024-04-18T22:26:38.210771+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:38.210771+0000","last_undegraded":"2024-04-18T22:26:38.210771+0000","last_fullsized":"2024-04-18T22:26:38.210771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:27.906774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338129+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:38.338129+0000","last_peered":"2024-04-18T22:26:38.338129+0000","last_clean":"2024-04-18T22:26:38.338129+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:38.338129+0000","last_undegraded":"2024-04-18T22:26:38.338129+0000","last_fullsized":"2024-04-18T22:26:38.338129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209335+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209335+0000","last_peered":"2024-04-18T22:26:38.209335+0000","last_clean":"2024-04-18T22:26:38.209335+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209335+0000","last_undegraded":"2024-04-18T22:26:38.209335+0000","last_fullsized":"2024-04-18T22:26:38.209335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:20:10.385885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210802+0000","last_change":"2024-04-18T22:26:26.053562+0000","last_active":"2024-04-18T22:26:38.210802+0000","last_peered":"2024-04-18T22:26:38.210802+0000","last_clean":"2024-04-18T22:26:38.210802+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:38.210802+0000","last_undegraded":"2024-04-18T22:26:38.210802+0000","last_fullsized":"2024-04-18T22:26:38.210802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:03.633713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340461+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:38.340461+0000","last_peered":"2024-04-18T22:26:38.340461+0000","last_clean":"2024-04-18T22:26:38.340461+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:38.340461+0000","last_undegraded":"2024-04-18T22:26:38.340461+0000","last_fullsized":"2024-04-18T22:26:38.340461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241369+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:38.241369+0000","last_peered":"2024-04-18T22:26:38.241369+0000","last_clean":"2024-04-18T22:26:38.241369+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:38.241369+0000","last_undegraded":"2024-04-18T22:26:38.241369+0000","last_fullsized":"2024-04-18T22:26:38.241369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:07.716048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211498+0000","last_change":"2024-04-18T22:26:26.057802+0000","last_active":"2024-04-18T22:26:38.211498+0000","last_peered":"2024-04-18T22:26:38.211498+0000","last_clean":"2024-04-18T22:26:38.211498+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:26:38.211498+0000","last_undegraded":"2024-04-18T22:26:38.211498+0000","last_fullsized":"2024-04-18T22:26:38.211498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:58:25.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339284+0000","last_change":"2024-04-18T22:26:34.217013+0000","last_active":"2024-04-18T22:26:38.339284+0000","last_peered":"2024-04-18T22:26:38.339284+0000","last_clean":"2024-04-18T22:26:38.339284+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:26:38.339284+0000","last_undegraded":"2024-04-18T22:26:38.339284+0000","last_fullsized":"2024-04-18T22:26:38.339284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:37.784925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210263+0000","last_change":"2024-04-18T22:26:26.050006+0000","last_active":"2024-04-18T22:26:38.210263+0000","last_peered":"2024-04-18T22:26:38.210263+0000","last_clean":"2024-04-18T22:26:38.210263+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:38.210263+0000","last_undegraded":"2024-04-18T22:26:38.210263+0000","last_fullsized":"2024-04-18T22:26:38.210263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:33:05.635452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338277+0000","last_change":"2024-04-18T22:26:26.060702+0000","last_active":"2024-04-18T22:26:38.338277+0000","last_peered":"2024-04-18T22:26:38.338277+0000","last_clean":"2024-04-18T22:26:38.338277+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:38.338277+0000","last_undegraded":"2024-04-18T22:26:38.338277+0000","last_fullsized":"2024-04-18T22:26:38.338277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:18:36.194967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239065+0000","last_change":"2024-04-18T22:26:37.250932+0000","last_active":"2024-04-18T22:26:38.239065+0000","last_peered":"2024-04-18T22:26:38.239065+0000","last_clean":"2024-04-18T22:26:38.239065+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:38.239065+0000","last_undegraded":"2024-04-18T22:26:38.239065+0000","last_fullsized":"2024-04-18T22:26:38.239065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:03.780060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210800+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210800+0000","last_peered":"2024-04-18T22:26:38.210800+0000","last_clean":"2024-04-18T22:26:38.210800+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210800+0000","last_undegraded":"2024-04-18T22:26:38.210800+0000","last_fullsized":"2024-04-18T22:26:38.210800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:29:28.068569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241235+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:38.241235+0000","last_peered":"2024-04-18T22:26:38.241235+0000","last_clean":"2024-04-18T22:26:38.241235+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:38.241235+0000","last_undegraded":"2024-04-18T22:26:38.241235+0000","last_fullsized":"2024-04-18T22:26:38.241235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:49:04.469351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211082+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211082+0000","last_peered":"2024-04-18T22:26:38.211082+0000","last_clean":"2024-04-18T22:26:38.211082+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211082+0000","last_undegraded":"2024-04-18T22:26:38.211082+0000","last_fullsized":"2024-04-18T22:26:38.211082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:49:20.072221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339128+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:26:38.339128+0000","last_peered":"2024-04-18T22:26:38.339128+0000","last_clean":"2024-04-18T22:26:38.339128+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:26:38.339128+0000","last_undegraded":"2024-04-18T22:26:38.339128+0000","last_fullsized":"2024-04-18T22:26:38.339128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340715+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340715+0000","last_peered":"2024-04-18T22:26:38.340715+0000","last_clean":"2024-04-18T22:26:38.340715+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340715+0000","last_undegraded":"2024-04-18T22:26:38.340715+0000","last_fullsized":"2024-04-18T22:26:38.340715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209937+0000","last_change":"2024-04-18T22:26:26.052492+0000","last_active":"2024-04-18T22:26:38.209937+0000","last_peered":"2024-04-18T22:26:38.209937+0000","last_clean":"2024-04-18T22:26:38.209937+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:38.209937+0000","last_undegraded":"2024-04-18T22:26:38.209937+0000","last_fullsized":"2024-04-18T22:26:38.209937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:39:51.990646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339820+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339820+0000","last_peered":"2024-04-18T22:26:38.339820+0000","last_clean":"2024-04-18T22:26:38.339820+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339820+0000","last_undegraded":"2024-04-18T22:26:38.339820+0000","last_fullsized":"2024-04-18T22:26:38.339820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:37.891211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239177+0000","last_change":"2024-04-18T22:26:37.250746+0000","last_active":"2024-04-18T22:26:38.239177+0000","last_peered":"2024-04-18T22:26:38.239177+0000","last_clean":"2024-04-18T22:26:38.239177+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:38.239177+0000","last_undegraded":"2024-04-18T22:26:38.239177+0000","last_fullsized":"2024-04-18T22:26:38.239177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:58:41.981858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240749+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:38.240749+0000","last_peered":"2024-04-18T22:26:38.240749+0000","last_clean":"2024-04-18T22:26:38.240749+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:38.240749+0000","last_undegraded":"2024-04-18T22:26:38.240749+0000","last_fullsized":"2024-04-18T22:26:38.240749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209349+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209349+0000","last_peered":"2024-04-18T22:26:38.209349+0000","last_clean":"2024-04-18T22:26:38.209349+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209349+0000","last_undegraded":"2024-04-18T22:26:38.209349+0000","last_fullsized":"2024-04-18T22:26:38.209349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:25:14.095942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210807+0000","last_change":"2024-04-18T22:26:26.056290+0000","last_active":"2024-04-18T22:26:38.210807+0000","last_peered":"2024-04-18T22:26:38.210807+0000","last_clean":"2024-04-18T22:26:38.210807+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:38.210807+0000","last_undegraded":"2024-04-18T22:26:38.210807+0000","last_fullsized":"2024-04-18T22:26:38.210807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:33:17.740653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209496+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209496+0000","last_peered":"2024-04-18T22:26:38.209496+0000","last_clean":"2024-04-18T22:26:38.209496+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209496+0000","last_undegraded":"2024-04-18T22:26:38.209496+0000","last_fullsized":"2024-04-18T22:26:38.209496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:02:53.362524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241052+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:38.241052+0000","last_peered":"2024-04-18T22:26:38.241052+0000","last_clean":"2024-04-18T22:26:38.241052+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:38.241052+0000","last_undegraded":"2024-04-18T22:26:38.241052+0000","last_fullsized":"2024-04-18T22:26:38.241052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:26.383475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239565+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:38.239565+0000","last_peered":"2024-04-18T22:26:38.239565+0000","last_clean":"2024-04-18T22:26:38.239565+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:38.239565+0000","last_undegraded":"2024-04-18T22:26:38.239565+0000","last_fullsized":"2024-04-18T22:26:38.239565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:44:05.966703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241128+0000","last_change":"2024-04-18T22:26:37.248320+0000","last_active":"2024-04-18T22:26:38.241128+0000","last_peered":"2024-04-18T22:26:38.241128+0000","last_clean":"2024-04-18T22:26:38.241128+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:38.241128+0000","last_undegraded":"2024-04-18T22:26:38.241128+0000","last_fullsized":"2024-04-18T22:26:38.241128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:36.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339542+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:38.339542+0000","last_peered":"2024-04-18T22:26:38.339542+0000","last_clean":"2024-04-18T22:26:38.339542+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:38.339542+0000","last_undegraded":"2024-04-18T22:26:38.339542+0000","last_fullsized":"2024-04-18T22:26:38.339542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339629+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339629+0000","last_peered":"2024-04-18T22:26:38.339629+0000","last_clean":"2024-04-18T22:26:38.339629+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339629+0000","last_undegraded":"2024-04-18T22:26:38.339629+0000","last_fullsized":"2024-04-18T22:26:38.339629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:30.374649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240601+0000","last_change":"2024-04-18T22:26:37.253099+0000","last_active":"2024-04-18T22:26:38.240601+0000","last_peered":"2024-04-18T22:26:38.240601+0000","last_clean":"2024-04-18T22:26:38.240601+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:38.240601+0000","last_undegraded":"2024-04-18T22:26:38.240601+0000","last_fullsized":"2024-04-18T22:26:38.240601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:08.780794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240100+0000","last_change":"2024-04-18T22:26:37.242866+0000","last_active":"2024-04-18T22:26:38.240100+0000","last_peered":"2024-04-18T22:26:38.240100+0000","last_clean":"2024-04-18T22:26:38.240100+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:26:38.240100+0000","last_undegraded":"2024-04-18T22:26:38.240100+0000","last_fullsized":"2024-04-18T22:26:38.240100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:36.007938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240704+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:38.240704+0000","last_peered":"2024-04-18T22:26:38.240704+0000","last_clean":"2024-04-18T22:26:38.240704+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:38.240704+0000","last_undegraded":"2024-04-18T22:26:38.240704+0000","last_fullsized":"2024-04-18T22:26:38.240704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:44:13.894111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209504+0000","last_change":"2024-04-18T22:26:26.052580+0000","last_active":"2024-04-18T22:26:38.209504+0000","last_peered":"2024-04-18T22:26:38.209504+0000","last_clean":"2024-04-18T22:26:38.209504+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:38.209504+0000","last_undegraded":"2024-04-18T22:26:38.209504+0000","last_fullsized":"2024-04-18T22:26:38.209504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:22:38.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240069+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:38.240069+0000","last_peered":"2024-04-18T22:26:38.240069+0000","last_clean":"2024-04-18T22:26:38.240069+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:38.240069+0000","last_undegraded":"2024-04-18T22:26:38.240069+0000","last_fullsized":"2024-04-18T22:26:38.240069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209668+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209668+0000","last_peered":"2024-04-18T22:26:38.209668+0000","last_clean":"2024-04-18T22:26:38.209668+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209668+0000","last_undegraded":"2024-04-18T22:26:38.209668+0000","last_fullsized":"2024-04-18T22:26:38.209668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:48:30.030678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340512+0000","last_change":"2024-04-18T22:26:26.070592+0000","last_active":"2024-04-18T22:26:38.340512+0000","last_peered":"2024-04-18T22:26:38.340512+0000","last_clean":"2024-04-18T22:26:38.340512+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:38.340512+0000","last_undegraded":"2024-04-18T22:26:38.340512+0000","last_fullsized":"2024-04-18T22:26:38.340512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:10:37.345485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240135+0000","last_change":"2024-04-18T22:26:37.247379+0000","last_active":"2024-04-18T22:26:38.240135+0000","last_peered":"2024-04-18T22:26:38.240135+0000","last_clean":"2024-04-18T22:26:38.240135+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:26:38.240135+0000","last_undegraded":"2024-04-18T22:26:38.240135+0000","last_fullsized":"2024-04-18T22:26:38.240135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:31.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241409+0000","last_change":"2024-04-18T22:26:37.251309+0000","last_active":"2024-04-18T22:26:38.241409+0000","last_peered":"2024-04-18T22:26:38.241409+0000","last_clean":"2024-04-18T22:26:38.241409+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:26:38.241409+0000","last_undegraded":"2024-04-18T22:26:38.241409+0000","last_fullsized":"2024-04-18T22:26:38.241409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:55:54.687547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338304+0000","last_change":"2024-04-18T22:26:34.216163+0000","last_active":"2024-04-18T22:26:38.338304+0000","last_peered":"2024-04-18T22:26:38.338304+0000","last_clean":"2024-04-18T22:26:38.338304+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:38.338304+0000","last_undegraded":"2024-04-18T22:26:38.338304+0000","last_fullsized":"2024-04-18T22:26:38.338304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:07.705933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241207+0000","last_change":"2024-04-18T22:26:37.253789+0000","last_active":"2024-04-18T22:26:38.241207+0000","last_peered":"2024-04-18T22:26:38.241207+0000","last_clean":"2024-04-18T22:26:38.241207+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:38.241207+0000","last_undegraded":"2024-04-18T22:26:38.241207+0000","last_fullsized":"2024-04-18T22:26:38.241207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:56.318449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240074+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:38.240074+0000","last_peered":"2024-04-18T22:26:38.240074+0000","last_clean":"2024-04-18T22:26:38.240074+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:38.240074+0000","last_undegraded":"2024-04-18T22:26:38.240074+0000","last_fullsized":"2024-04-18T22:26:38.240074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:51:03.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338880+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:26:38.338880+0000","last_peered":"2024-04-18T22:26:38.338880+0000","last_clean":"2024-04-18T22:26:38.338880+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:26:38.338880+0000","last_undegraded":"2024-04-18T22:26:38.338880+0000","last_fullsized":"2024-04-18T22:26:38.338880+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211260+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211260+0000","last_peered":"2024-04-18T22:26:38.211260+0000","last_clean":"2024-04-18T22:26:38.211260+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211260+0000","last_undegraded":"2024-04-18T22:26:38.211260+0000","last_fullsized":"2024-04-18T22:26:38.211260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:40.351753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339220+0000","last_change":"2024-04-18T22:26:26.067550+0000","last_active":"2024-04-18T22:26:38.339220+0000","last_peered":"2024-04-18T22:26:38.339220+0000","last_clean":"2024-04-18T22:26:38.339220+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:38.339220+0000","last_undegraded":"2024-04-18T22:26:38.339220+0000","last_fullsized":"2024-04-18T22:26:38.339220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:49:44.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210088+0000","last_change":"2024-04-18T22:26:26.058882+0000","last_active":"2024-04-18T22:26:38.210088+0000","last_peered":"2024-04-18T22:26:38.210088+0000","last_clean":"2024-04-18T22:26:38.210088+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:38.210088+0000","last_undegraded":"2024-04-18T22:26:38.210088+0000","last_fullsized":"2024-04-18T22:26:38.210088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:47:23.720729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340082+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340082+0000","last_peered":"2024-04-18T22:26:38.340082+0000","last_clean":"2024-04-18T22:26:38.340082+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340082+0000","last_undegraded":"2024-04-18T22:26:38.340082+0000","last_fullsized":"2024-04-18T22:26:38.340082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240361+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:38.240361+0000","last_peered":"2024-04-18T22:26:38.240361+0000","last_clean":"2024-04-18T22:26:38.240361+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:38.240361+0000","last_undegraded":"2024-04-18T22:26:38.240361+0000","last_fullsized":"2024-04-18T22:26:38.240361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:32.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241182+0000","last_change":"2024-04-18T22:26:37.247834+0000","last_active":"2024-04-18T22:26:38.241182+0000","last_peered":"2024-04-18T22:26:38.241182+0000","last_clean":"2024-04-18T22:26:38.241182+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:38.241182+0000","last_undegraded":"2024-04-18T22:26:38.241182+0000","last_fullsized":"2024-04-18T22:26:38.241182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:45:58.922778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339394+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339394+0000","last_peered":"2024-04-18T22:26:38.339394+0000","last_clean":"2024-04-18T22:26:38.339394+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339394+0000","last_undegraded":"2024-04-18T22:26:38.339394+0000","last_fullsized":"2024-04-18T22:26:38.339394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240376+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:38.240376+0000","last_peered":"2024-04-18T22:26:38.240376+0000","last_clean":"2024-04-18T22:26:38.240376+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:38.240376+0000","last_undegraded":"2024-04-18T22:26:38.240376+0000","last_fullsized":"2024-04-18T22:26:38.240376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:12:44.602848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340352+0000","last_change":"2024-04-18T22:26:26.068751+0000","last_active":"2024-04-18T22:26:38.340352+0000","last_peered":"2024-04-18T22:26:38.340352+0000","last_clean":"2024-04-18T22:26:38.340352+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:38.340352+0000","last_undegraded":"2024-04-18T22:26:38.340352+0000","last_fullsized":"2024-04-18T22:26:38.340352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:58.461110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210605+0000","last_change":"2024-04-18T22:26:26.050076+0000","last_active":"2024-04-18T22:26:38.210605+0000","last_peered":"2024-04-18T22:26:38.210605+0000","last_clean":"2024-04-18T22:26:38.210605+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:38.210605+0000","last_undegraded":"2024-04-18T22:26:38.210605+0000","last_fullsized":"2024-04-18T22:26:38.210605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:43:48.685468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339247+0000","last_change":"2024-04-18T22:26:34.214565+0000","last_active":"2024-04-18T22:26:38.339247+0000","last_peered":"2024-04-18T22:26:38.339247+0000","last_clean":"2024-04-18T22:26:38.339247+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:38.339247+0000","last_undegraded":"2024-04-18T22:26:38.339247+0000","last_fullsized":"2024-04-18T22:26:38.339247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:00:17.620775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211798+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211798+0000","last_peered":"2024-04-18T22:26:38.211798+0000","last_clean":"2024-04-18T22:26:38.211798+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211798+0000","last_undegraded":"2024-04-18T22:26:38.211798+0000","last_fullsized":"2024-04-18T22:26:38.211798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:41.505834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209934+0000","last_change":"2024-04-18T22:26:26.049861+0000","last_active":"2024-04-18T22:26:38.209934+0000","last_peered":"2024-04-18T22:26:38.209934+0000","last_clean":"2024-04-18T22:26:38.209934+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:38.209934+0000","last_undegraded":"2024-04-18T22:26:38.209934+0000","last_fullsized":"2024-04-18T22:26:38.209934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:36:09.243064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340992+0000","last_change":"2024-04-18T22:26:26.071169+0000","last_active":"2024-04-18T22:26:38.340992+0000","last_peered":"2024-04-18T22:26:38.340992+0000","last_clean":"2024-04-18T22:26:38.340992+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:38.340992+0000","last_undegraded":"2024-04-18T22:26:38.340992+0000","last_fullsized":"2024-04-18T22:26:38.340992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:28.018224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339892+0000","last_change":"2024-04-18T22:26:34.216022+0000","last_active":"2024-04-18T22:26:38.339892+0000","last_peered":"2024-04-18T22:26:38.339892+0000","last_clean":"2024-04-18T22:26:38.339892+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:38.339892+0000","last_undegraded":"2024-04-18T22:26:38.339892+0000","last_fullsized":"2024-04-18T22:26:38.339892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:11:39.518803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338036+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338036+0000","last_peered":"2024-04-18T22:26:38.338036+0000","last_clean":"2024-04-18T22:26:38.338036+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338036+0000","last_undegraded":"2024-04-18T22:26:38.338036+0000","last_fullsized":"2024-04-18T22:26:38.338036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:37.018548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340605+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340605+0000","last_peered":"2024-04-18T22:26:38.340605+0000","last_clean":"2024-04-18T22:26:38.340605+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340605+0000","last_undegraded":"2024-04-18T22:26:38.340605+0000","last_fullsized":"2024-04-18T22:26:38.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:31.938261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240827+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:38.240827+0000","last_peered":"2024-04-18T22:26:38.240827+0000","last_clean":"2024-04-18T22:26:38.240827+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:38.240827+0000","last_undegraded":"2024-04-18T22:26:38.240827+0000","last_fullsized":"2024-04-18T22:26:38.240827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:39.646164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210238+0000","last_change":"2024-04-18T22:26:26.049399+0000","last_active":"2024-04-18T22:26:38.210238+0000","last_peered":"2024-04-18T22:26:38.210238+0000","last_clean":"2024-04-18T22:26:38.210238+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:26:38.210238+0000","last_undegraded":"2024-04-18T22:26:38.210238+0000","last_fullsized":"2024-04-18T22:26:38.210238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:58:33.909228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240079+0000","last_change":"2024-04-18T22:26:37.253737+0000","last_active":"2024-04-18T22:26:38.240079+0000","last_peered":"2024-04-18T22:26:38.240079+0000","last_clean":"2024-04-18T22:26:38.240079+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:38.240079+0000","last_undegraded":"2024-04-18T22:26:38.240079+0000","last_fullsized":"2024-04-18T22:26:38.240079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:58:47.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240898+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.240898+0000","last_peered":"2024-04-18T22:26:38.240898+0000","last_clean":"2024-04-18T22:26:38.240898+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.240898+0000","last_undegraded":"2024-04-18T22:26:38.240898+0000","last_fullsized":"2024-04-18T22:26:38.240898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:33.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338838+0000","last_change":"2024-04-18T22:26:26.059340+0000","last_active":"2024-04-18T22:26:38.338838+0000","last_peered":"2024-04-18T22:26:38.338838+0000","last_clean":"2024-04-18T22:26:38.338838+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:26:38.338838+0000","last_undegraded":"2024-04-18T22:26:38.338838+0000","last_fullsized":"2024-04-18T22:26:38.338838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:53:02.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211394+0000","last_change":"2024-04-18T22:26:26.058539+0000","last_active":"2024-04-18T22:26:38.211394+0000","last_peered":"2024-04-18T22:26:38.211394+0000","last_clean":"2024-04-18T22:26:38.211394+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:38.211394+0000","last_undegraded":"2024-04-18T22:26:38.211394+0000","last_fullsized":"2024-04-18T22:26:38.211394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:29.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240762+0000","last_change":"2024-04-18T22:26:37.245223+0000","last_active":"2024-04-18T22:26:38.240762+0000","last_peered":"2024-04-18T22:26:38.240762+0000","last_clean":"2024-04-18T22:26:38.240762+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:26:38.240762+0000","last_undegraded":"2024-04-18T22:26:38.240762+0000","last_fullsized":"2024-04-18T22:26:38.240762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:06:32.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338415+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338415+0000","last_peered":"2024-04-18T22:26:38.338415+0000","last_clean":"2024-04-18T22:26:38.338415+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338415+0000","last_undegraded":"2024-04-18T22:26:38.338415+0000","last_fullsized":"2024-04-18T22:26:38.338415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340762+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340762+0000","last_peered":"2024-04-18T22:26:38.340762+0000","last_clean":"2024-04-18T22:26:38.340762+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340762+0000","last_undegraded":"2024-04-18T22:26:38.340762+0000","last_fullsized":"2024-04-18T22:26:38.340762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338370+0000","last_change":"2024-04-18T22:26:34.221491+0000","last_active":"2024-04-18T22:26:38.338370+0000","last_peered":"2024-04-18T22:26:38.338370+0000","last_clean":"2024-04-18T22:26:38.338370+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:38.338370+0000","last_undegraded":"2024-04-18T22:26:38.338370+0000","last_fullsized":"2024-04-18T22:26:38.338370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211095+0000","last_change":"2024-04-18T22:26:26.053516+0000","last_active":"2024-04-18T22:26:38.211095+0000","last_peered":"2024-04-18T22:26:38.211095+0000","last_clean":"2024-04-18T22:26:38.211095+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:38.211095+0000","last_undegraded":"2024-04-18T22:26:38.211095+0000","last_fullsized":"2024-04-18T22:26:38.211095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:27:51.463983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338110+0000","last_change":"2024-04-18T22:26:34.215787+0000","last_active":"2024-04-18T22:26:38.338110+0000","last_peered":"2024-04-18T22:26:38.338110+0000","last_clean":"2024-04-18T22:26:38.338110+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:38.338110+0000","last_undegraded":"2024-04-18T22:26:38.338110+0000","last_fullsized":"2024-04-18T22:26:38.338110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:32.105080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340172+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:26:38.340172+0000","last_peered":"2024-04-18T22:26:38.340172+0000","last_clean":"2024-04-18T22:26:38.340172+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:26:38.340172+0000","last_undegraded":"2024-04-18T22:26:38.340172+0000","last_fullsized":"2024-04-18T22:26:38.340172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338576+0000","last_change":"2024-04-18T22:26:34.218064+0000","last_active":"2024-04-18T22:26:38.338576+0000","last_peered":"2024-04-18T22:26:38.338576+0000","last_clean":"2024-04-18T22:26:38.338576+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:26:38.338576+0000","last_undegraded":"2024-04-18T22:26:38.338576+0000","last_fullsized":"2024-04-18T22:26:38.338576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211199+0000","last_change":"2024-04-18T22:26:26.058084+0000","last_active":"2024-04-18T22:26:38.211199+0000","last_peered":"2024-04-18T22:26:38.211199+0000","last_clean":"2024-04-18T22:26:38.211199+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:26:38.211199+0000","last_undegraded":"2024-04-18T22:26:38.211199+0000","last_fullsized":"2024-04-18T22:26:38.211199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:53:45.480811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209942+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.209942+0000","last_peered":"2024-04-18T22:26:38.209942+0000","last_clean":"2024-04-18T22:26:38.209942+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.209942+0000","last_undegraded":"2024-04-18T22:26:38.209942+0000","last_fullsized":"2024-04-18T22:26:38.209942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:06:09.947309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211063+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211063+0000","last_peered":"2024-04-18T22:26:38.211063+0000","last_clean":"2024-04-18T22:26:38.211063+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211063+0000","last_undegraded":"2024-04-18T22:26:38.211063+0000","last_fullsized":"2024-04-18T22:26:38.211063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:00.276319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209992+0000","last_change":"2024-04-18T22:26:26.049503+0000","last_active":"2024-04-18T22:26:38.209992+0000","last_peered":"2024-04-18T22:26:38.209992+0000","last_clean":"2024-04-18T22:26:38.209992+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:38.209992+0000","last_undegraded":"2024-04-18T22:26:38.209992+0000","last_fullsized":"2024-04-18T22:26:38.209992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:47:34.371634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239668+0000","last_change":"2024-04-18T22:26:37.254175+0000","last_active":"2024-04-18T22:26:38.239668+0000","last_peered":"2024-04-18T22:26:38.239668+0000","last_clean":"2024-04-18T22:26:38.239668+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:38.239668+0000","last_undegraded":"2024-04-18T22:26:38.239668+0000","last_fullsized":"2024-04-18T22:26:38.239668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:48:00.575406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240707+0000","last_change":"2024-04-18T22:26:37.253978+0000","last_active":"2024-04-18T22:26:38.240707+0000","last_peered":"2024-04-18T22:26:38.240707+0000","last_clean":"2024-04-18T22:26:38.240707+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:38.240707+0000","last_undegraded":"2024-04-18T22:26:38.240707+0000","last_fullsized":"2024-04-18T22:26:38.240707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:38.579496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241140+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:38.241140+0000","last_peered":"2024-04-18T22:26:38.241140+0000","last_clean":"2024-04-18T22:26:38.241140+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:38.241140+0000","last_undegraded":"2024-04-18T22:26:38.241140+0000","last_fullsized":"2024-04-18T22:26:38.241140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:40:37.721198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210913+0000","last_change":"2024-04-18T22:26:26.054055+0000","last_active":"2024-04-18T22:26:38.210913+0000","last_peered":"2024-04-18T22:26:38.210913+0000","last_clean":"2024-04-18T22:26:38.210913+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:38.210913+0000","last_undegraded":"2024-04-18T22:26:38.210913+0000","last_fullsized":"2024-04-18T22:26:38.210913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:13.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209807+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209807+0000","last_peered":"2024-04-18T22:26:38.209807+0000","last_clean":"2024-04-18T22:26:38.209807+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209807+0000","last_undegraded":"2024-04-18T22:26:38.209807+0000","last_fullsized":"2024-04-18T22:26:38.209807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:26.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340548+0000","last_change":"2024-04-18T22:26:26.069294+0000","last_active":"2024-04-18T22:26:38.340548+0000","last_peered":"2024-04-18T22:26:38.340548+0000","last_clean":"2024-04-18T22:26:38.340548+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:38.340548+0000","last_undegraded":"2024-04-18T22:26:38.340548+0000","last_fullsized":"2024-04-18T22:26:38.340548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:46.805559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239221+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239221+0000","last_peered":"2024-04-18T22:26:38.239221+0000","last_clean":"2024-04-18T22:26:38.239221+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239221+0000","last_undegraded":"2024-04-18T22:26:38.239221+0000","last_fullsized":"2024-04-18T22:26:38.239221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339975+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.339975+0000","last_peered":"2024-04-18T22:26:38.339975+0000","last_clean":"2024-04-18T22:26:38.339975+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.339975+0000","last_undegraded":"2024-04-18T22:26:38.339975+0000","last_fullsized":"2024-04-18T22:26:38.339975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:02.769450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738425,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078668,"statfs":{"total":96636764160,"available":96336556032,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547205,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334752,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037088,"statfs":{"total":96636764160,"available":96293978112,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:26:44.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:26:44.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:44.562 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:26:38.231275+0000","last_up_change":"2024-04-18T22:26:25.018371+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":35,"up_from":75,"up_thru":78,"down_at":36,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6801","nonce":1652764673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6803","nonce":1652764673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6807","nonce":1652764673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6805","nonce":1652764673}]},"public_addr":"172.21.15.96:6801/1652764673","cluster_addr":"172.21.15.96:6803/1652764673","heartbeat_back_addr":"172.21.15.96:6807/1652764673","heartbeat_front_addr":"172.21.15.96:6805/1652764673","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":35},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:26:45.580 INFO:tasks.ceph:Scrubbing osd.0 2024-04-18T22:26:45.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-18T22:26:45.747 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:26:45.748 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T22:26:45.748 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:26:45.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-18T22:26:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2024-04-18T22:26:46.080 INFO:tasks.ceph:Scrubbing osd.1 2024-04-18T22:26:46.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-18T22:26:46.227 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:26:46.228 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-18T22:26:46.228 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:26:46.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-18T22:26:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2024-04-18T22:26:46.539 INFO:tasks.ceph:Scrubbing osd.2 2024-04-18T22:26:46.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-18T22:26:46.684 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-18T22:26:46.684 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T22:26:46.684 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-18T22:26:46.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-18T22:26:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2024-04-18T22:26:47.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:26:47.309 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:26:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:26:47.326 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-04-18T22:26:46.950396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669824,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445696,"statfs":{"total":289910292480,"available":289224392704,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.731280"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211682+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211682+0000","last_peered":"2024-04-18T22:26:38.211682+0000","last_clean":"2024-04-18T22:26:38.211682+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211682+0000","last_undegraded":"2024-04-18T22:26:38.211682+0000","last_fullsized":"2024-04-18T22:26:38.211682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:16:02.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211107+0000","last_change":"2024-04-18T22:26:26.036531+0000","last_active":"2024-04-18T22:26:38.211107+0000","last_peered":"2024-04-18T22:26:38.211107+0000","last_clean":"2024-04-18T22:26:38.211107+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:26:38.211107+0000","last_undegraded":"2024-04-18T22:26:38.211107+0000","last_fullsized":"2024-04-18T22:26:38.211107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:17:26.168838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241543+0000","last_change":"2024-04-18T22:26:37.250608+0000","last_active":"2024-04-18T22:26:38.241543+0000","last_peered":"2024-04-18T22:26:38.241543+0000","last_clean":"2024-04-18T22:26:38.241543+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:38.241543+0000","last_undegraded":"2024-04-18T22:26:38.241543+0000","last_fullsized":"2024-04-18T22:26:38.241543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:32.948871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240445+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:38.240445+0000","last_peered":"2024-04-18T22:26:38.240445+0000","last_clean":"2024-04-18T22:26:38.240445+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:38.240445+0000","last_undegraded":"2024-04-18T22:26:38.240445+0000","last_fullsized":"2024-04-18T22:26:38.240445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:26:07.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210220+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210220+0000","last_peered":"2024-04-18T22:26:38.210220+0000","last_clean":"2024-04-18T22:26:38.210220+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210220+0000","last_undegraded":"2024-04-18T22:26:38.210220+0000","last_fullsized":"2024-04-18T22:26:38.210220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:35:56.119968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239263+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239263+0000","last_peered":"2024-04-18T22:26:38.239263+0000","last_clean":"2024-04-18T22:26:38.239263+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239263+0000","last_undegraded":"2024-04-18T22:26:38.239263+0000","last_fullsized":"2024-04-18T22:26:38.239263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:04:33.941262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239350+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:38.239350+0000","last_peered":"2024-04-18T22:26:38.239350+0000","last_clean":"2024-04-18T22:26:38.239350+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:38.239350+0000","last_undegraded":"2024-04-18T22:26:38.239350+0000","last_fullsized":"2024-04-18T22:26:38.239350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:21:05.317874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238671+0000","last_change":"2024-04-18T22:26:37.251857+0000","last_active":"2024-04-18T22:26:38.238671+0000","last_peered":"2024-04-18T22:26:38.238671+0000","last_clean":"2024-04-18T22:26:38.238671+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:26:38.238671+0000","last_undegraded":"2024-04-18T22:26:38.238671+0000","last_fullsized":"2024-04-18T22:26:38.238671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:07.544858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340474+0000","last_change":"2024-04-18T22:26:26.069968+0000","last_active":"2024-04-18T22:26:38.340474+0000","last_peered":"2024-04-18T22:26:38.340474+0000","last_clean":"2024-04-18T22:26:38.340474+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:38.340474+0000","last_undegraded":"2024-04-18T22:26:38.340474+0000","last_fullsized":"2024-04-18T22:26:38.340474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:16:05.689716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211534+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211534+0000","last_peered":"2024-04-18T22:26:38.211534+0000","last_clean":"2024-04-18T22:26:38.211534+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211534+0000","last_undegraded":"2024-04-18T22:26:38.211534+0000","last_fullsized":"2024-04-18T22:26:38.211534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:43:12.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241442+0000","last_change":"2024-04-18T22:26:37.250594+0000","last_active":"2024-04-18T22:26:38.241442+0000","last_peered":"2024-04-18T22:26:38.241442+0000","last_clean":"2024-04-18T22:26:38.241442+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:38.241442+0000","last_undegraded":"2024-04-18T22:26:38.241442+0000","last_fullsized":"2024-04-18T22:26:38.241442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:31.385259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211588+0000","last_change":"2024-04-18T22:26:26.057510+0000","last_active":"2024-04-18T22:26:38.211588+0000","last_peered":"2024-04-18T22:26:38.211588+0000","last_clean":"2024-04-18T22:26:38.211588+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:38.211588+0000","last_undegraded":"2024-04-18T22:26:38.211588+0000","last_fullsized":"2024-04-18T22:26:38.211588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:10.312012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210691+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.210691+0000","last_peered":"2024-04-18T22:26:38.210691+0000","last_clean":"2024-04-18T22:26:38.210691+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.210691+0000","last_undegraded":"2024-04-18T22:26:38.210691+0000","last_fullsized":"2024-04-18T22:26:38.210691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:45:48.363681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239408+0000","last_change":"2024-04-18T22:26:37.253632+0000","last_active":"2024-04-18T22:26:38.239408+0000","last_peered":"2024-04-18T22:26:38.239408+0000","last_clean":"2024-04-18T22:26:38.239408+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:38.239408+0000","last_undegraded":"2024-04-18T22:26:38.239408+0000","last_fullsized":"2024-04-18T22:26:38.239408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:28:37.568125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239299+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239299+0000","last_peered":"2024-04-18T22:26:38.239299+0000","last_clean":"2024-04-18T22:26:38.239299+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239299+0000","last_undegraded":"2024-04-18T22:26:38.239299+0000","last_fullsized":"2024-04-18T22:26:38.239299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:08.716543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.238847+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:38.238847+0000","last_peered":"2024-04-18T22:26:38.238847+0000","last_clean":"2024-04-18T22:26:38.238847+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:38.238847+0000","last_undegraded":"2024-04-18T22:26:38.238847+0000","last_fullsized":"2024-04-18T22:26:38.238847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:50.499991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340902+0000","last_change":"2024-04-18T22:26:26.070529+0000","last_active":"2024-04-18T22:26:38.340902+0000","last_peered":"2024-04-18T22:26:38.340902+0000","last_clean":"2024-04-18T22:26:38.340902+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:38.340902+0000","last_undegraded":"2024-04-18T22:26:38.340902+0000","last_fullsized":"2024-04-18T22:26:38.340902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:52:17.813661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210997+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.210997+0000","last_peered":"2024-04-18T22:26:38.210997+0000","last_clean":"2024-04-18T22:26:38.210997+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.210997+0000","last_undegraded":"2024-04-18T22:26:38.210997+0000","last_fullsized":"2024-04-18T22:26:38.210997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:25.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339371+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339371+0000","last_peered":"2024-04-18T22:26:38.339371+0000","last_clean":"2024-04-18T22:26:38.339371+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339371+0000","last_undegraded":"2024-04-18T22:26:38.339371+0000","last_fullsized":"2024-04-18T22:26:38.339371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:42.038434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240563+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:38.240563+0000","last_peered":"2024-04-18T22:26:38.240563+0000","last_clean":"2024-04-18T22:26:38.240563+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:38.240563+0000","last_undegraded":"2024-04-18T22:26:38.240563+0000","last_fullsized":"2024-04-18T22:26:38.240563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:51:13.814270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240871+0000","last_change":"2024-04-18T22:26:37.248780+0000","last_active":"2024-04-18T22:26:38.240871+0000","last_peered":"2024-04-18T22:26:38.240871+0000","last_clean":"2024-04-18T22:26:38.240871+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:38.240871+0000","last_undegraded":"2024-04-18T22:26:38.240871+0000","last_fullsized":"2024-04-18T22:26:38.240871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210771+0000","last_change":"2024-04-18T22:26:26.049675+0000","last_active":"2024-04-18T22:26:38.210771+0000","last_peered":"2024-04-18T22:26:38.210771+0000","last_clean":"2024-04-18T22:26:38.210771+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:38.210771+0000","last_undegraded":"2024-04-18T22:26:38.210771+0000","last_fullsized":"2024-04-18T22:26:38.210771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:27.906774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338129+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:38.338129+0000","last_peered":"2024-04-18T22:26:38.338129+0000","last_clean":"2024-04-18T22:26:38.338129+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:38.338129+0000","last_undegraded":"2024-04-18T22:26:38.338129+0000","last_fullsized":"2024-04-18T22:26:38.338129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:15:40.495145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209335+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209335+0000","last_peered":"2024-04-18T22:26:38.209335+0000","last_clean":"2024-04-18T22:26:38.209335+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209335+0000","last_undegraded":"2024-04-18T22:26:38.209335+0000","last_fullsized":"2024-04-18T22:26:38.209335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:20:10.385885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210802+0000","last_change":"2024-04-18T22:26:26.053562+0000","last_active":"2024-04-18T22:26:38.210802+0000","last_peered":"2024-04-18T22:26:38.210802+0000","last_clean":"2024-04-18T22:26:38.210802+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:38.210802+0000","last_undegraded":"2024-04-18T22:26:38.210802+0000","last_fullsized":"2024-04-18T22:26:38.210802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:32:03.633713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340461+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:38.340461+0000","last_peered":"2024-04-18T22:26:38.340461+0000","last_clean":"2024-04-18T22:26:38.340461+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:38.340461+0000","last_undegraded":"2024-04-18T22:26:38.340461+0000","last_fullsized":"2024-04-18T22:26:38.340461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:19:13.640794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241369+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:38.241369+0000","last_peered":"2024-04-18T22:26:38.241369+0000","last_clean":"2024-04-18T22:26:38.241369+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:38.241369+0000","last_undegraded":"2024-04-18T22:26:38.241369+0000","last_fullsized":"2024-04-18T22:26:38.241369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:07.716048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211498+0000","last_change":"2024-04-18T22:26:26.057802+0000","last_active":"2024-04-18T22:26:38.211498+0000","last_peered":"2024-04-18T22:26:38.211498+0000","last_clean":"2024-04-18T22:26:38.211498+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:26:38.211498+0000","last_undegraded":"2024-04-18T22:26:38.211498+0000","last_fullsized":"2024-04-18T22:26:38.211498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:58:25.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339284+0000","last_change":"2024-04-18T22:26:34.217013+0000","last_active":"2024-04-18T22:26:38.339284+0000","last_peered":"2024-04-18T22:26:38.339284+0000","last_clean":"2024-04-18T22:26:38.339284+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:26:38.339284+0000","last_undegraded":"2024-04-18T22:26:38.339284+0000","last_fullsized":"2024-04-18T22:26:38.339284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:37.784925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210263+0000","last_change":"2024-04-18T22:26:26.050006+0000","last_active":"2024-04-18T22:26:38.210263+0000","last_peered":"2024-04-18T22:26:38.210263+0000","last_clean":"2024-04-18T22:26:38.210263+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:38.210263+0000","last_undegraded":"2024-04-18T22:26:38.210263+0000","last_fullsized":"2024-04-18T22:26:38.210263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:33:05.635452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338430+0000","last_change":"2024-04-18T22:26:26.060702+0000","last_active":"2024-04-18T22:26:38.338430+0000","last_peered":"2024-04-18T22:26:38.338430+0000","last_clean":"2024-04-18T22:26:38.338430+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:38.338430+0000","last_undegraded":"2024-04-18T22:26:38.338430+0000","last_fullsized":"2024-04-18T22:26:38.338430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:18:36.194967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239065+0000","last_change":"2024-04-18T22:26:37.250932+0000","last_active":"2024-04-18T22:26:38.239065+0000","last_peered":"2024-04-18T22:26:38.239065+0000","last_clean":"2024-04-18T22:26:38.239065+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:38.239065+0000","last_undegraded":"2024-04-18T22:26:38.239065+0000","last_fullsized":"2024-04-18T22:26:38.239065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:03.780060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210800+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210800+0000","last_peered":"2024-04-18T22:26:38.210800+0000","last_clean":"2024-04-18T22:26:38.210800+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210800+0000","last_undegraded":"2024-04-18T22:26:38.210800+0000","last_fullsized":"2024-04-18T22:26:38.210800+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:29:28.068569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241235+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:38.241235+0000","last_peered":"2024-04-18T22:26:38.241235+0000","last_clean":"2024-04-18T22:26:38.241235+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:38.241235+0000","last_undegraded":"2024-04-18T22:26:38.241235+0000","last_fullsized":"2024-04-18T22:26:38.241235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:49:04.469351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211082+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211082+0000","last_peered":"2024-04-18T22:26:38.211082+0000","last_clean":"2024-04-18T22:26:38.211082+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211082+0000","last_undegraded":"2024-04-18T22:26:38.211082+0000","last_fullsized":"2024-04-18T22:26:38.211082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:49:20.072221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339128+0000","last_change":"2024-04-18T22:22:31.140990+0000","last_active":"2024-04-18T22:26:38.339128+0000","last_peered":"2024-04-18T22:26:38.339128+0000","last_clean":"2024-04-18T22:26:38.339128+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:26:38.339128+0000","last_undegraded":"2024-04-18T22:26:38.339128+0000","last_fullsized":"2024-04-18T22:26:38.339128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:53:36.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340715+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340715+0000","last_peered":"2024-04-18T22:26:38.340715+0000","last_clean":"2024-04-18T22:26:38.340715+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340715+0000","last_undegraded":"2024-04-18T22:26:38.340715+0000","last_fullsized":"2024-04-18T22:26:38.340715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:55:40.975682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209937+0000","last_change":"2024-04-18T22:26:26.052492+0000","last_active":"2024-04-18T22:26:38.209937+0000","last_peered":"2024-04-18T22:26:38.209937+0000","last_clean":"2024-04-18T22:26:38.209937+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:38.209937+0000","last_undegraded":"2024-04-18T22:26:38.209937+0000","last_fullsized":"2024-04-18T22:26:38.209937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:39:51.990646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339820+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339820+0000","last_peered":"2024-04-18T22:26:38.339820+0000","last_clean":"2024-04-18T22:26:38.339820+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339820+0000","last_undegraded":"2024-04-18T22:26:38.339820+0000","last_fullsized":"2024-04-18T22:26:38.339820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:37.891211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"17'8","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239177+0000","last_change":"2024-04-18T22:26:37.250746+0000","last_active":"2024-04-18T22:26:38.239177+0000","last_peered":"2024-04-18T22:26:38.239177+0000","last_clean":"2024-04-18T22:26:38.239177+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:38.239177+0000","last_undegraded":"2024-04-18T22:26:38.239177+0000","last_fullsized":"2024-04-18T22:26:38.239177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:58:41.981858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240749+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:38.240749+0000","last_peered":"2024-04-18T22:26:38.240749+0000","last_clean":"2024-04-18T22:26:38.240749+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:38.240749+0000","last_undegraded":"2024-04-18T22:26:38.240749+0000","last_fullsized":"2024-04-18T22:26:38.240749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209349+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209349+0000","last_peered":"2024-04-18T22:26:38.209349+0000","last_clean":"2024-04-18T22:26:38.209349+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209349+0000","last_undegraded":"2024-04-18T22:26:38.209349+0000","last_fullsized":"2024-04-18T22:26:38.209349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:25:14.095942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210807+0000","last_change":"2024-04-18T22:26:26.056290+0000","last_active":"2024-04-18T22:26:38.210807+0000","last_peered":"2024-04-18T22:26:38.210807+0000","last_clean":"2024-04-18T22:26:38.210807+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:38.210807+0000","last_undegraded":"2024-04-18T22:26:38.210807+0000","last_fullsized":"2024-04-18T22:26:38.210807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:33:17.740653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209496+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209496+0000","last_peered":"2024-04-18T22:26:38.209496+0000","last_clean":"2024-04-18T22:26:38.209496+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209496+0000","last_undegraded":"2024-04-18T22:26:38.209496+0000","last_fullsized":"2024-04-18T22:26:38.209496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:02:53.362524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241052+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:38.241052+0000","last_peered":"2024-04-18T22:26:38.241052+0000","last_clean":"2024-04-18T22:26:38.241052+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:38.241052+0000","last_undegraded":"2024-04-18T22:26:38.241052+0000","last_fullsized":"2024-04-18T22:26:38.241052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:15:26.383475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239565+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:38.239565+0000","last_peered":"2024-04-18T22:26:38.239565+0000","last_clean":"2024-04-18T22:26:38.239565+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:38.239565+0000","last_undegraded":"2024-04-18T22:26:38.239565+0000","last_fullsized":"2024-04-18T22:26:38.239565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:44:05.966703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241128+0000","last_change":"2024-04-18T22:26:37.248320+0000","last_active":"2024-04-18T22:26:38.241128+0000","last_peered":"2024-04-18T22:26:38.241128+0000","last_clean":"2024-04-18T22:26:38.241128+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:38.241128+0000","last_undegraded":"2024-04-18T22:26:38.241128+0000","last_fullsized":"2024-04-18T22:26:38.241128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:36.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339542+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:38.339542+0000","last_peered":"2024-04-18T22:26:38.339542+0000","last_clean":"2024-04-18T22:26:38.339542+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:38.339542+0000","last_undegraded":"2024-04-18T22:26:38.339542+0000","last_fullsized":"2024-04-18T22:26:38.339542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:21:49.422364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339629+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339629+0000","last_peered":"2024-04-18T22:26:38.339629+0000","last_clean":"2024-04-18T22:26:38.339629+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339629+0000","last_undegraded":"2024-04-18T22:26:38.339629+0000","last_fullsized":"2024-04-18T22:26:38.339629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:30.374649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240601+0000","last_change":"2024-04-18T22:26:37.253099+0000","last_active":"2024-04-18T22:26:38.240601+0000","last_peered":"2024-04-18T22:26:38.240601+0000","last_clean":"2024-04-18T22:26:38.240601+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:38.240601+0000","last_undegraded":"2024-04-18T22:26:38.240601+0000","last_fullsized":"2024-04-18T22:26:38.240601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:08.780794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240100+0000","last_change":"2024-04-18T22:26:37.242866+0000","last_active":"2024-04-18T22:26:38.240100+0000","last_peered":"2024-04-18T22:26:38.240100+0000","last_clean":"2024-04-18T22:26:38.240100+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:26:38.240100+0000","last_undegraded":"2024-04-18T22:26:38.240100+0000","last_fullsized":"2024-04-18T22:26:38.240100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:36.007938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240704+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:38.240704+0000","last_peered":"2024-04-18T22:26:38.240704+0000","last_clean":"2024-04-18T22:26:38.240704+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:38.240704+0000","last_undegraded":"2024-04-18T22:26:38.240704+0000","last_fullsized":"2024-04-18T22:26:38.240704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:44:13.894111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209504+0000","last_change":"2024-04-18T22:26:26.052580+0000","last_active":"2024-04-18T22:26:38.209504+0000","last_peered":"2024-04-18T22:26:38.209504+0000","last_clean":"2024-04-18T22:26:38.209504+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:38.209504+0000","last_undegraded":"2024-04-18T22:26:38.209504+0000","last_fullsized":"2024-04-18T22:26:38.209504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:22:38.757624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240069+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:38.240069+0000","last_peered":"2024-04-18T22:26:38.240069+0000","last_clean":"2024-04-18T22:26:38.240069+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:38.240069+0000","last_undegraded":"2024-04-18T22:26:38.240069+0000","last_fullsized":"2024-04-18T22:26:38.240069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209668+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209668+0000","last_peered":"2024-04-18T22:26:38.209668+0000","last_clean":"2024-04-18T22:26:38.209668+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209668+0000","last_undegraded":"2024-04-18T22:26:38.209668+0000","last_fullsized":"2024-04-18T22:26:38.209668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:48:30.030678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340512+0000","last_change":"2024-04-18T22:26:26.070592+0000","last_active":"2024-04-18T22:26:38.340512+0000","last_peered":"2024-04-18T22:26:38.340512+0000","last_clean":"2024-04-18T22:26:38.340512+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:38.340512+0000","last_undegraded":"2024-04-18T22:26:38.340512+0000","last_fullsized":"2024-04-18T22:26:38.340512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:10:37.345485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'157","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240135+0000","last_change":"2024-04-18T22:26:37.247379+0000","last_active":"2024-04-18T22:26:38.240135+0000","last_peered":"2024-04-18T22:26:38.240135+0000","last_clean":"2024-04-18T22:26:38.240135+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:26:38.240135+0000","last_undegraded":"2024-04-18T22:26:38.240135+0000","last_fullsized":"2024-04-18T22:26:38.240135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:20.901738+0000","last_clean_scrub_stamp":"2024-04-18T22:22:20.901738+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:20:31.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241409+0000","last_change":"2024-04-18T22:26:37.251309+0000","last_active":"2024-04-18T22:26:38.241409+0000","last_peered":"2024-04-18T22:26:38.241409+0000","last_clean":"2024-04-18T22:26:38.241409+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:26:38.241409+0000","last_undegraded":"2024-04-18T22:26:38.241409+0000","last_fullsized":"2024-04-18T22:26:38.241409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:55:54.687547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338304+0000","last_change":"2024-04-18T22:26:34.216163+0000","last_active":"2024-04-18T22:26:38.338304+0000","last_peered":"2024-04-18T22:26:38.338304+0000","last_clean":"2024-04-18T22:26:38.338304+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:38.338304+0000","last_undegraded":"2024-04-18T22:26:38.338304+0000","last_fullsized":"2024-04-18T22:26:38.338304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:38:07.705933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241316+0000","last_change":"2024-04-18T22:26:37.253789+0000","last_active":"2024-04-18T22:26:38.241316+0000","last_peered":"2024-04-18T22:26:38.241316+0000","last_clean":"2024-04-18T22:26:38.241316+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:38.241316+0000","last_undegraded":"2024-04-18T22:26:38.241316+0000","last_fullsized":"2024-04-18T22:26:38.241316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:56.318449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240074+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:38.240074+0000","last_peered":"2024-04-18T22:26:38.240074+0000","last_clean":"2024-04-18T22:26:38.240074+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:38.240074+0000","last_undegraded":"2024-04-18T22:26:38.240074+0000","last_fullsized":"2024-04-18T22:26:38.240074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:51:03.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339006+0000","last_change":"2024-04-18T22:22:31.137984+0000","last_active":"2024-04-18T22:26:38.339006+0000","last_peered":"2024-04-18T22:26:38.339006+0000","last_clean":"2024-04-18T22:26:38.339006+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:26:38.339006+0000","last_undegraded":"2024-04-18T22:26:38.339006+0000","last_fullsized":"2024-04-18T22:26:38.339006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:57:32.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211260+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211260+0000","last_peered":"2024-04-18T22:26:38.211260+0000","last_clean":"2024-04-18T22:26:38.211260+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211260+0000","last_undegraded":"2024-04-18T22:26:38.211260+0000","last_fullsized":"2024-04-18T22:26:38.211260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:40.351753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339220+0000","last_change":"2024-04-18T22:26:26.067550+0000","last_active":"2024-04-18T22:26:38.339220+0000","last_peered":"2024-04-18T22:26:38.339220+0000","last_clean":"2024-04-18T22:26:38.339220+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:38.339220+0000","last_undegraded":"2024-04-18T22:26:38.339220+0000","last_fullsized":"2024-04-18T22:26:38.339220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:49:44.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210088+0000","last_change":"2024-04-18T22:26:26.058882+0000","last_active":"2024-04-18T22:26:38.210088+0000","last_peered":"2024-04-18T22:26:38.210088+0000","last_clean":"2024-04-18T22:26:38.210088+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:38.210088+0000","last_undegraded":"2024-04-18T22:26:38.210088+0000","last_fullsized":"2024-04-18T22:26:38.210088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:47:23.720729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340082+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340082+0000","last_peered":"2024-04-18T22:26:38.340082+0000","last_clean":"2024-04-18T22:26:38.340082+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340082+0000","last_undegraded":"2024-04-18T22:26:38.340082+0000","last_fullsized":"2024-04-18T22:26:38.340082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:14.039236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240361+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:38.240361+0000","last_peered":"2024-04-18T22:26:38.240361+0000","last_clean":"2024-04-18T22:26:38.240361+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:38.240361+0000","last_undegraded":"2024-04-18T22:26:38.240361+0000","last_fullsized":"2024-04-18T22:26:38.240361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:00:32.757545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241182+0000","last_change":"2024-04-18T22:26:37.247834+0000","last_active":"2024-04-18T22:26:38.241182+0000","last_peered":"2024-04-18T22:26:38.241182+0000","last_clean":"2024-04-18T22:26:38.241182+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:38.241182+0000","last_undegraded":"2024-04-18T22:26:38.241182+0000","last_fullsized":"2024-04-18T22:26:38.241182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:45:58.922778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339394+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339394+0000","last_peered":"2024-04-18T22:26:38.339394+0000","last_clean":"2024-04-18T22:26:38.339394+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339394+0000","last_undegraded":"2024-04-18T22:26:38.339394+0000","last_fullsized":"2024-04-18T22:26:38.339394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:18:09.333101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240376+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:38.240376+0000","last_peered":"2024-04-18T22:26:38.240376+0000","last_clean":"2024-04-18T22:26:38.240376+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:38.240376+0000","last_undegraded":"2024-04-18T22:26:38.240376+0000","last_fullsized":"2024-04-18T22:26:38.240376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:12:44.602848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340352+0000","last_change":"2024-04-18T22:26:26.068751+0000","last_active":"2024-04-18T22:26:38.340352+0000","last_peered":"2024-04-18T22:26:38.340352+0000","last_clean":"2024-04-18T22:26:38.340352+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:38.340352+0000","last_undegraded":"2024-04-18T22:26:38.340352+0000","last_fullsized":"2024-04-18T22:26:38.340352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:58.461110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210605+0000","last_change":"2024-04-18T22:26:26.050076+0000","last_active":"2024-04-18T22:26:38.210605+0000","last_peered":"2024-04-18T22:26:38.210605+0000","last_clean":"2024-04-18T22:26:38.210605+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:38.210605+0000","last_undegraded":"2024-04-18T22:26:38.210605+0000","last_fullsized":"2024-04-18T22:26:38.210605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:43:48.685468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339247+0000","last_change":"2024-04-18T22:26:34.214565+0000","last_active":"2024-04-18T22:26:38.339247+0000","last_peered":"2024-04-18T22:26:38.339247+0000","last_clean":"2024-04-18T22:26:38.339247+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:38.339247+0000","last_undegraded":"2024-04-18T22:26:38.339247+0000","last_fullsized":"2024-04-18T22:26:38.339247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:00:17.620775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211798+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211798+0000","last_peered":"2024-04-18T22:26:38.211798+0000","last_clean":"2024-04-18T22:26:38.211798+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211798+0000","last_undegraded":"2024-04-18T22:26:38.211798+0000","last_fullsized":"2024-04-18T22:26:38.211798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:41.505834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209934+0000","last_change":"2024-04-18T22:26:26.049861+0000","last_active":"2024-04-18T22:26:38.209934+0000","last_peered":"2024-04-18T22:26:38.209934+0000","last_clean":"2024-04-18T22:26:38.209934+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:38.209934+0000","last_undegraded":"2024-04-18T22:26:38.209934+0000","last_fullsized":"2024-04-18T22:26:38.209934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:36:09.243064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"17'2","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340992+0000","last_change":"2024-04-18T22:26:26.071169+0000","last_active":"2024-04-18T22:26:38.340992+0000","last_peered":"2024-04-18T22:26:38.340992+0000","last_clean":"2024-04-18T22:26:38.340992+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:38.340992+0000","last_undegraded":"2024-04-18T22:26:38.340992+0000","last_fullsized":"2024-04-18T22:26:38.340992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:28.018224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339892+0000","last_change":"2024-04-18T22:26:34.216022+0000","last_active":"2024-04-18T22:26:38.339892+0000","last_peered":"2024-04-18T22:26:38.339892+0000","last_clean":"2024-04-18T22:26:38.339892+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:38.339892+0000","last_undegraded":"2024-04-18T22:26:38.339892+0000","last_fullsized":"2024-04-18T22:26:38.339892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:11:39.518803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338036+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338036+0000","last_peered":"2024-04-18T22:26:38.338036+0000","last_clean":"2024-04-18T22:26:38.338036+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338036+0000","last_undegraded":"2024-04-18T22:26:38.338036+0000","last_fullsized":"2024-04-18T22:26:38.338036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:49:37.018548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340605+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340605+0000","last_peered":"2024-04-18T22:26:38.340605+0000","last_clean":"2024-04-18T22:26:38.340605+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340605+0000","last_undegraded":"2024-04-18T22:26:38.340605+0000","last_fullsized":"2024-04-18T22:26:38.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:32:31.938261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240827+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:38.240827+0000","last_peered":"2024-04-18T22:26:38.240827+0000","last_clean":"2024-04-18T22:26:38.240827+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:38.240827+0000","last_undegraded":"2024-04-18T22:26:38.240827+0000","last_fullsized":"2024-04-18T22:26:38.240827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:39.646164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210238+0000","last_change":"2024-04-18T22:26:26.049399+0000","last_active":"2024-04-18T22:26:38.210238+0000","last_peered":"2024-04-18T22:26:38.210238+0000","last_clean":"2024-04-18T22:26:38.210238+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:26:38.210238+0000","last_undegraded":"2024-04-18T22:26:38.210238+0000","last_fullsized":"2024-04-18T22:26:38.210238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:58:33.909228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240079+0000","last_change":"2024-04-18T22:26:37.253737+0000","last_active":"2024-04-18T22:26:38.240079+0000","last_peered":"2024-04-18T22:26:38.240079+0000","last_clean":"2024-04-18T22:26:38.240079+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:38.240079+0000","last_undegraded":"2024-04-18T22:26:38.240079+0000","last_fullsized":"2024-04-18T22:26:38.240079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:58:47.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240898+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.240898+0000","last_peered":"2024-04-18T22:26:38.240898+0000","last_clean":"2024-04-18T22:26:38.240898+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.240898+0000","last_undegraded":"2024-04-18T22:26:38.240898+0000","last_fullsized":"2024-04-18T22:26:38.240898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:02:33.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338838+0000","last_change":"2024-04-18T22:26:26.059340+0000","last_active":"2024-04-18T22:26:38.338838+0000","last_peered":"2024-04-18T22:26:38.338838+0000","last_clean":"2024-04-18T22:26:38.338838+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:26:38.338838+0000","last_undegraded":"2024-04-18T22:26:38.338838+0000","last_fullsized":"2024-04-18T22:26:38.338838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:53:02.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"17'3","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211394+0000","last_change":"2024-04-18T22:26:26.058539+0000","last_active":"2024-04-18T22:26:38.211394+0000","last_peered":"2024-04-18T22:26:38.211394+0000","last_clean":"2024-04-18T22:26:38.211394+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:38.211394+0000","last_undegraded":"2024-04-18T22:26:38.211394+0000","last_fullsized":"2024-04-18T22:26:38.211394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:29.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240762+0000","last_change":"2024-04-18T22:26:37.245223+0000","last_active":"2024-04-18T22:26:38.240762+0000","last_peered":"2024-04-18T22:26:38.240762+0000","last_clean":"2024-04-18T22:26:38.240762+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:26:38.240762+0000","last_undegraded":"2024-04-18T22:26:38.240762+0000","last_fullsized":"2024-04-18T22:26:38.240762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:06:32.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338415+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338415+0000","last_peered":"2024-04-18T22:26:38.338415+0000","last_clean":"2024-04-18T22:26:38.338415+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338415+0000","last_undegraded":"2024-04-18T22:26:38.338415+0000","last_fullsized":"2024-04-18T22:26:38.338415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:46:46.861959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340762+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340762+0000","last_peered":"2024-04-18T22:26:38.340762+0000","last_clean":"2024-04-18T22:26:38.340762+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340762+0000","last_undegraded":"2024-04-18T22:26:38.340762+0000","last_fullsized":"2024-04-18T22:26:38.340762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:44:29.842227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338370+0000","last_change":"2024-04-18T22:26:34.221491+0000","last_active":"2024-04-18T22:26:38.338370+0000","last_peered":"2024-04-18T22:26:38.338370+0000","last_clean":"2024-04-18T22:26:38.338370+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:38.338370+0000","last_undegraded":"2024-04-18T22:26:38.338370+0000","last_fullsized":"2024-04-18T22:26:38.338370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211095+0000","last_change":"2024-04-18T22:26:26.053516+0000","last_active":"2024-04-18T22:26:38.211095+0000","last_peered":"2024-04-18T22:26:38.211095+0000","last_clean":"2024-04-18T22:26:38.211095+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:38.211095+0000","last_undegraded":"2024-04-18T22:26:38.211095+0000","last_fullsized":"2024-04-18T22:26:38.211095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:27:51.463983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338110+0000","last_change":"2024-04-18T22:26:34.215787+0000","last_active":"2024-04-18T22:26:38.338110+0000","last_peered":"2024-04-18T22:26:38.338110+0000","last_clean":"2024-04-18T22:26:38.338110+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:38.338110+0000","last_undegraded":"2024-04-18T22:26:38.338110+0000","last_fullsized":"2024-04-18T22:26:38.338110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:46:32.105080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340172+0000","last_change":"2024-04-18T22:22:32.144765+0000","last_active":"2024-04-18T22:26:38.340172+0000","last_peered":"2024-04-18T22:26:38.340172+0000","last_clean":"2024-04-18T22:26:38.340172+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:26:38.340172+0000","last_undegraded":"2024-04-18T22:26:38.340172+0000","last_fullsized":"2024-04-18T22:26:38.340172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:19.210172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338576+0000","last_change":"2024-04-18T22:26:34.218064+0000","last_active":"2024-04-18T22:26:38.338576+0000","last_peered":"2024-04-18T22:26:38.338576+0000","last_clean":"2024-04-18T22:26:38.338576+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:26:38.338576+0000","last_undegraded":"2024-04-18T22:26:38.338576+0000","last_fullsized":"2024-04-18T22:26:38.338576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:02:03.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211199+0000","last_change":"2024-04-18T22:26:26.058084+0000","last_active":"2024-04-18T22:26:38.211199+0000","last_peered":"2024-04-18T22:26:38.211199+0000","last_clean":"2024-04-18T22:26:38.211199+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:26:38.211199+0000","last_undegraded":"2024-04-18T22:26:38.211199+0000","last_fullsized":"2024-04-18T22:26:38.211199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:53:45.480811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209942+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.209942+0000","last_peered":"2024-04-18T22:26:38.209942+0000","last_clean":"2024-04-18T22:26:38.209942+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.209942+0000","last_undegraded":"2024-04-18T22:26:38.209942+0000","last_fullsized":"2024-04-18T22:26:38.209942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:06:09.947309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211063+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211063+0000","last_peered":"2024-04-18T22:26:38.211063+0000","last_clean":"2024-04-18T22:26:38.211063+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211063+0000","last_undegraded":"2024-04-18T22:26:38.211063+0000","last_fullsized":"2024-04-18T22:26:38.211063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:26:00.276319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209992+0000","last_change":"2024-04-18T22:26:26.049503+0000","last_active":"2024-04-18T22:26:38.209992+0000","last_peered":"2024-04-18T22:26:38.209992+0000","last_clean":"2024-04-18T22:26:38.209992+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:38.209992+0000","last_undegraded":"2024-04-18T22:26:38.209992+0000","last_fullsized":"2024-04-18T22:26:38.209992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:47:34.371634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239668+0000","last_change":"2024-04-18T22:26:37.254175+0000","last_active":"2024-04-18T22:26:38.239668+0000","last_peered":"2024-04-18T22:26:38.239668+0000","last_clean":"2024-04-18T22:26:38.239668+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:38.239668+0000","last_undegraded":"2024-04-18T22:26:38.239668+0000","last_fullsized":"2024-04-18T22:26:38.239668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:48:00.575406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"17'2","reported_seq":95,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.240707+0000","last_change":"2024-04-18T22:26:37.253978+0000","last_active":"2024-04-18T22:26:38.240707+0000","last_peered":"2024-04-18T22:26:38.240707+0000","last_clean":"2024-04-18T22:26:38.240707+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:38.240707+0000","last_undegraded":"2024-04-18T22:26:38.240707+0000","last_fullsized":"2024-04-18T22:26:38.240707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:52:38.579496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241140+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:38.241140+0000","last_peered":"2024-04-18T22:26:38.241140+0000","last_clean":"2024-04-18T22:26:38.241140+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:38.241140+0000","last_undegraded":"2024-04-18T22:26:38.241140+0000","last_fullsized":"2024-04-18T22:26:38.241140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:40:37.721198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"17'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210913+0000","last_change":"2024-04-18T22:26:26.054055+0000","last_active":"2024-04-18T22:26:38.210913+0000","last_peered":"2024-04-18T22:26:38.210913+0000","last_clean":"2024-04-18T22:26:38.210913+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:38.210913+0000","last_undegraded":"2024-04-18T22:26:38.210913+0000","last_fullsized":"2024-04-18T22:26:38.210913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:16:13.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209807+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209807+0000","last_peered":"2024-04-18T22:26:38.209807+0000","last_clean":"2024-04-18T22:26:38.209807+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209807+0000","last_undegraded":"2024-04-18T22:26:38.209807+0000","last_fullsized":"2024-04-18T22:26:38.209807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:50:26.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340548+0000","last_change":"2024-04-18T22:26:26.069294+0000","last_active":"2024-04-18T22:26:38.340548+0000","last_peered":"2024-04-18T22:26:38.340548+0000","last_clean":"2024-04-18T22:26:38.340548+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:38.340548+0000","last_undegraded":"2024-04-18T22:26:38.340548+0000","last_fullsized":"2024-04-18T22:26:38.340548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:46.805559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239221+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239221+0000","last_peered":"2024-04-18T22:26:38.239221+0000","last_clean":"2024-04-18T22:26:38.239221+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239221+0000","last_undegraded":"2024-04-18T22:26:38.239221+0000","last_fullsized":"2024-04-18T22:26:38.239221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:02:06.238546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339975+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.339975+0000","last_peered":"2024-04-18T22:26:38.339975+0000","last_clean":"2024-04-18T22:26:38.339975+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.339975+0000","last_undegraded":"2024-04-18T22:26:38.339975+0000","last_fullsized":"2024-04-18T22:26:38.339975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:21:02.769450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738426,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078668,"statfs":{"total":96636764160,"available":96336556032,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547206,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334752,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037088,"statfs":{"total":96636764160,"available":96293978112,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:26:47.333 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.333 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.334 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.335 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.336 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.337 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.338 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-18T22:22:20.901738+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=20, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.339 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.340 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.341 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.342 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.343 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:26:47.344 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-18T22:27:07.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:27:07.660 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:27:07.662 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:27:07.677 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-04-18T22:27:06.957588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669792,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445728,"statfs":{"total":289910292480,"available":289224425472,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004289"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211797+0000","last_change":"2024-04-18T22:26:34.196887+0000","last_active":"2024-04-18T22:26:38.211797+0000","last_peered":"2024-04-18T22:26:38.211797+0000","last_clean":"2024-04-18T22:26:38.211797+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:26:38.211797+0000","last_undegraded":"2024-04-18T22:26:38.211797+0000","last_fullsized":"2024-04-18T22:26:38.211797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:03.702263+0000","last_change":"2024-04-18T22:27:03.702317+0000","last_active":"2024-04-18T22:27:03.702263+0000","last_peered":"2024-04-18T22:27:03.702263+0000","last_clean":"2024-04-18T22:27:03.702263+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:27:03.702263+0000","last_undegraded":"2024-04-18T22:27:03.702263+0000","last_fullsized":"2024-04-18T22:27:03.702263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:03.702062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:03.702062+0000","last_clean_scrub_stamp":"2024-04-18T22:27:03.702062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:16:32.180267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:57.764363+0000","last_change":"2024-04-18T22:26:57.764480+0000","last_active":"2024-04-18T22:26:57.764363+0000","last_peered":"2024-04-18T22:26:57.764363+0000","last_clean":"2024-04-18T22:26:57.764363+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:57.764363+0000","last_undegraded":"2024-04-18T22:26:57.764363+0000","last_fullsized":"2024-04-18T22:26:57.764363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:57.764113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:57.764113+0000","last_clean_scrub_stamp":"2024-04-18T22:26:57.764113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:21:34.665656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.052029+0000","last_change":"2024-04-18T22:26:37.254777+0000","last_active":"2024-04-18T22:26:39.052029+0000","last_peered":"2024-04-18T22:26:39.052029+0000","last_clean":"2024-04-18T22:26:39.052029+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:26:39.052029+0000","last_undegraded":"2024-04-18T22:26:39.052029+0000","last_fullsized":"2024-04-18T22:26:39.052029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210367+0000","last_change":"2024-04-18T22:26:26.034204+0000","last_active":"2024-04-18T22:26:38.210367+0000","last_peered":"2024-04-18T22:26:38.210367+0000","last_clean":"2024-04-18T22:26:38.210367+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:26:38.210367+0000","last_undegraded":"2024-04-18T22:26:38.210367+0000","last_fullsized":"2024-04-18T22:26:38.210367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":92,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239416+0000","last_change":"2024-04-18T22:26:37.254351+0000","last_active":"2024-04-18T22:26:38.239416+0000","last_peered":"2024-04-18T22:26:38.239416+0000","last_clean":"2024-04-18T22:26:38.239416+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:26:38.239416+0000","last_undegraded":"2024-04-18T22:26:38.239416+0000","last_fullsized":"2024-04-18T22:26:38.239416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.051858+0000","last_change":"2024-04-18T22:26:37.238873+0000","last_active":"2024-04-18T22:26:39.051858+0000","last_peered":"2024-04-18T22:26:39.051858+0000","last_clean":"2024-04-18T22:26:39.051858+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:26:39.051858+0000","last_undegraded":"2024-04-18T22:26:39.051858+0000","last_fullsized":"2024-04-18T22:26:39.051858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:03.794704+0000","last_change":"2024-04-18T22:27:03.794755+0000","last_active":"2024-04-18T22:27:03.794704+0000","last_peered":"2024-04-18T22:27:03.794704+0000","last_clean":"2024-04-18T22:27:03.794704+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:27:03.794704+0000","last_undegraded":"2024-04-18T22:27:03.794704+0000","last_fullsized":"2024-04-18T22:27:03.794704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:03.794514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:03.794514+0000","last_clean_scrub_stamp":"2024-04-18T22:27:03.794514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:18.073591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:51.223495+0000","last_change":"2024-04-18T22:26:51.223538+0000","last_active":"2024-04-18T22:26:51.223495+0000","last_peered":"2024-04-18T22:26:51.223495+0000","last_clean":"2024-04-18T22:26:51.223495+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:51.223495+0000","last_undegraded":"2024-04-18T22:26:51.223495+0000","last_fullsized":"2024-04-18T22:26:51.223495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:51.223340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:51.223340+0000","last_clean_scrub_stamp":"2024-04-18T22:26:51.223340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:30:27.396252+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211650+0000","last_change":"2024-04-18T22:26:34.196457+0000","last_active":"2024-04-18T22:26:38.211650+0000","last_peered":"2024-04-18T22:26:38.211650+0000","last_clean":"2024-04-18T22:26:38.211650+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:26:38.211650+0000","last_undegraded":"2024-04-18T22:26:38.211650+0000","last_fullsized":"2024-04-18T22:26:38.211650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:53.711807+0000","last_change":"2024-04-18T22:26:53.711861+0000","last_active":"2024-04-18T22:26:53.711807+0000","last_peered":"2024-04-18T22:26:53.711807+0000","last_clean":"2024-04-18T22:26:53.711807+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:53.711807+0000","last_undegraded":"2024-04-18T22:26:53.711807+0000","last_fullsized":"2024-04-18T22:26:53.711807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:53.711620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:53.711620+0000","last_clean_scrub_stamp":"2024-04-18T22:26:53.711620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:29:43.095832+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.774267+0000","last_change":"2024-04-18T22:26:58.774324+0000","last_active":"2024-04-18T22:26:58.774267+0000","last_peered":"2024-04-18T22:26:58.774267+0000","last_clean":"2024-04-18T22:26:58.774267+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:58.774267+0000","last_undegraded":"2024-04-18T22:26:58.774267+0000","last_fullsized":"2024-04-18T22:26:58.774267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.774079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.774079+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.774079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:50:35.726437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211489+0000","last_change":"2024-04-18T22:26:34.187011+0000","last_active":"2024-04-18T22:26:38.211489+0000","last_peered":"2024-04-18T22:26:38.211489+0000","last_clean":"2024-04-18T22:26:38.211489+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:26:38.211489+0000","last_undegraded":"2024-04-18T22:26:38.211489+0000","last_fullsized":"2024-04-18T22:26:38.211489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:50.685335+0000","last_change":"2024-04-18T22:26:50.685388+0000","last_active":"2024-04-18T22:26:50.685335+0000","last_peered":"2024-04-18T22:26:50.685335+0000","last_clean":"2024-04-18T22:26:50.685335+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:50.685335+0000","last_undegraded":"2024-04-18T22:26:50.685335+0000","last_fullsized":"2024-04-18T22:26:50.685335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:50.685135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:50.685135+0000","last_clean_scrub_stamp":"2024-04-18T22:26:50.685135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:01:58.721958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239431+0000","last_change":"2024-04-18T22:26:37.231152+0000","last_active":"2024-04-18T22:26:38.239431+0000","last_peered":"2024-04-18T22:26:38.239431+0000","last_clean":"2024-04-18T22:26:38.239431+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:26:38.239431+0000","last_undegraded":"2024-04-18T22:26:38.239431+0000","last_fullsized":"2024-04-18T22:26:38.239431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.051755+0000","last_change":"2024-04-18T22:26:37.251877+0000","last_active":"2024-04-18T22:26:39.051755+0000","last_peered":"2024-04-18T22:26:39.051755+0000","last_clean":"2024-04-18T22:26:39.051755+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:26:39.051755+0000","last_undegraded":"2024-04-18T22:26:39.051755+0000","last_fullsized":"2024-04-18T22:26:39.051755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:49.193390+0000","last_change":"2024-04-18T22:26:49.193412+0000","last_active":"2024-04-18T22:26:49.193390+0000","last_peered":"2024-04-18T22:26:49.193390+0000","last_clean":"2024-04-18T22:26:49.193390+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:49.193390+0000","last_undegraded":"2024-04-18T22:26:49.193390+0000","last_fullsized":"2024-04-18T22:26:49.193390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:49.193291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:49.193291+0000","last_clean_scrub_stamp":"2024-04-18T22:26:49.193291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:56:57.689686+0000","scrub_duration":119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211120+0000","last_change":"2024-04-18T22:26:26.036254+0000","last_active":"2024-04-18T22:26:38.211120+0000","last_peered":"2024-04-18T22:26:38.211120+0000","last_clean":"2024-04-18T22:26:38.211120+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:26:38.211120+0000","last_undegraded":"2024-04-18T22:26:38.211120+0000","last_fullsized":"2024-04-18T22:26:38.211120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339486+0000","last_change":"2024-04-18T22:26:34.225030+0000","last_active":"2024-04-18T22:26:38.339486+0000","last_peered":"2024-04-18T22:26:38.339486+0000","last_clean":"2024-04-18T22:26:38.339486+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:26:38.339486+0000","last_undegraded":"2024-04-18T22:26:38.339486+0000","last_fullsized":"2024-04-18T22:26:38.339486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.052047+0000","last_change":"2024-04-18T22:26:37.254627+0000","last_active":"2024-04-18T22:26:39.052047+0000","last_peered":"2024-04-18T22:26:39.052047+0000","last_clean":"2024-04-18T22:26:39.052047+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:26:39.052047+0000","last_undegraded":"2024-04-18T22:26:39.052047+0000","last_fullsized":"2024-04-18T22:26:39.052047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:54.694230+0000","last_change":"2024-04-18T22:26:54.694275+0000","last_active":"2024-04-18T22:26:54.694230+0000","last_peered":"2024-04-18T22:26:54.694230+0000","last_clean":"2024-04-18T22:26:54.694230+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:54.694230+0000","last_undegraded":"2024-04-18T22:26:54.694230+0000","last_fullsized":"2024-04-18T22:26:54.694230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:54.694060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:54.694060+0000","last_clean_scrub_stamp":"2024-04-18T22:26:54.694060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:41:59.294287+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:57.788683+0000","last_change":"2024-04-18T22:26:57.788736+0000","last_active":"2024-04-18T22:26:57.788683+0000","last_peered":"2024-04-18T22:26:57.788683+0000","last_clean":"2024-04-18T22:26:57.788683+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:57.788683+0000","last_undegraded":"2024-04-18T22:26:57.788683+0000","last_fullsized":"2024-04-18T22:26:57.788683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:57.788490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:57.788490+0000","last_clean_scrub_stamp":"2024-04-18T22:26:57.788490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:17:54.948384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.064786+0000","last_change":"2024-04-18T22:22:32.143975+0000","last_active":"2024-04-18T22:26:46.064786+0000","last_peered":"2024-04-18T22:26:46.064786+0000","last_clean":"2024-04-18T22:26:46.064786+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:26:46.064786+0000","last_undegraded":"2024-04-18T22:26:46.064786+0000","last_fullsized":"2024-04-18T22:26:46.064786+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209472+0000","last_change":"2024-04-18T22:26:26.036033+0000","last_active":"2024-04-18T22:26:38.209472+0000","last_peered":"2024-04-18T22:26:38.209472+0000","last_clean":"2024-04-18T22:26:38.209472+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:26:38.209472+0000","last_undegraded":"2024-04-18T22:26:38.209472+0000","last_fullsized":"2024-04-18T22:26:38.209472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:52.934833+0000","last_change":"2024-04-18T22:26:52.934890+0000","last_active":"2024-04-18T22:26:52.934833+0000","last_peered":"2024-04-18T22:26:52.934833+0000","last_clean":"2024-04-18T22:26:52.934833+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:52.934833+0000","last_undegraded":"2024-04-18T22:26:52.934833+0000","last_fullsized":"2024-04-18T22:26:52.934833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:52.934621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:52.934621+0000","last_clean_scrub_stamp":"2024-04-18T22:26:52.934621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:35:31.513357+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.064831+0000","last_change":"2024-04-18T22:22:32.144403+0000","last_active":"2024-04-18T22:26:46.064831+0000","last_peered":"2024-04-18T22:26:46.064831+0000","last_clean":"2024-04-18T22:26:46.064831+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:26:46.064831+0000","last_undegraded":"2024-04-18T22:26:46.064831+0000","last_fullsized":"2024-04-18T22:26:46.064831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":114,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.992930+0000","last_change":"2024-04-18T22:26:37.256718+0000","last_active":"2024-04-18T22:26:46.992930+0000","last_peered":"2024-04-18T22:26:46.992930+0000","last_clean":"2024-04-18T22:26:46.992930+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:26:46.992930+0000","last_undegraded":"2024-04-18T22:26:46.992930+0000","last_fullsized":"2024-04-18T22:26:46.992930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.741215+0000","last_change":"2024-04-18T22:27:00.741353+0000","last_active":"2024-04-18T22:27:00.741215+0000","last_peered":"2024-04-18T22:27:00.741215+0000","last_clean":"2024-04-18T22:27:00.741215+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:27:00.741215+0000","last_undegraded":"2024-04-18T22:27:00.741215+0000","last_fullsized":"2024-04-18T22:27:00.741215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:00.740983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:00.740983+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.740983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:27:43.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.018418+0000","last_change":"2024-04-18T22:27:00.018443+0000","last_active":"2024-04-18T22:27:00.018418+0000","last_peered":"2024-04-18T22:27:00.018418+0000","last_clean":"2024-04-18T22:27:00.018418+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:27:00.018418+0000","last_undegraded":"2024-04-18T22:27:00.018418+0000","last_fullsized":"2024-04-18T22:27:00.018418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:00.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:00.018318+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.018318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:23:42.928114+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:53.876074+0000","last_change":"2024-04-18T22:26:53.876130+0000","last_active":"2024-04-18T22:26:53.876074+0000","last_peered":"2024-04-18T22:26:53.876074+0000","last_clean":"2024-04-18T22:26:53.876074+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:53.876074+0000","last_undegraded":"2024-04-18T22:26:53.876074+0000","last_fullsized":"2024-04-18T22:26:53.876074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:53.875887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:53.875887+0000","last_clean_scrub_stamp":"2024-04-18T22:26:53.875887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:17:28.614068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.995800+0000","last_change":"2024-04-18T22:26:58.995856+0000","last_active":"2024-04-18T22:26:58.995800+0000","last_peered":"2024-04-18T22:26:58.995800+0000","last_clean":"2024-04-18T22:26:58.995800+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:58.995800+0000","last_undegraded":"2024-04-18T22:26:58.995800+0000","last_fullsized":"2024-04-18T22:26:58.995800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.995601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.995601+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.995601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:51:48.807197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:59.762552+0000","last_change":"2024-04-18T22:26:59.762603+0000","last_active":"2024-04-18T22:26:59.762552+0000","last_peered":"2024-04-18T22:26:59.762552+0000","last_clean":"2024-04-18T22:26:59.762552+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:59.762552+0000","last_undegraded":"2024-04-18T22:26:59.762552+0000","last_fullsized":"2024-04-18T22:26:59.762552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:59.762355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:59.762355+0000","last_clean_scrub_stamp":"2024-04-18T22:26:59.762355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:46:46.389631+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210921+0000","last_change":"2024-04-18T22:26:34.195749+0000","last_active":"2024-04-18T22:26:38.210921+0000","last_peered":"2024-04-18T22:26:38.210921+0000","last_clean":"2024-04-18T22:26:38.210921+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:26:38.210921+0000","last_undegraded":"2024-04-18T22:26:38.210921+0000","last_fullsized":"2024-04-18T22:26:38.210921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"17'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.051982+0000","last_change":"2024-04-18T22:26:37.253003+0000","last_active":"2024-04-18T22:26:39.051982+0000","last_peered":"2024-04-18T22:26:39.051982+0000","last_clean":"2024-04-18T22:26:39.051982+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:26:39.051982+0000","last_undegraded":"2024-04-18T22:26:39.051982+0000","last_fullsized":"2024-04-18T22:26:39.051982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211217+0000","last_change":"2024-04-18T22:26:26.034287+0000","last_active":"2024-04-18T22:26:38.211217+0000","last_peered":"2024-04-18T22:26:38.211217+0000","last_clean":"2024-04-18T22:26:38.211217+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:26:38.211217+0000","last_undegraded":"2024-04-18T22:26:38.211217+0000","last_fullsized":"2024-04-18T22:26:38.211217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.981955+0000","last_change":"2024-04-18T22:27:04.982016+0000","last_active":"2024-04-18T22:27:04.981955+0000","last_peered":"2024-04-18T22:27:04.981955+0000","last_clean":"2024-04-18T22:27:04.981955+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:27:04.981955+0000","last_undegraded":"2024-04-18T22:27:04.981955+0000","last_fullsized":"2024-04-18T22:27:04.981955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.981759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.981759+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.981759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:45:49.051876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340780+0000","last_change":"2024-04-18T22:22:32.144132+0000","last_active":"2024-04-18T22:26:38.340780+0000","last_peered":"2024-04-18T22:26:38.340780+0000","last_clean":"2024-04-18T22:26:38.340780+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:26:38.340780+0000","last_undegraded":"2024-04-18T22:26:38.340780+0000","last_fullsized":"2024-04-18T22:26:38.340780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:49.902373+0000","last_change":"2024-04-18T22:26:49.902426+0000","last_active":"2024-04-18T22:26:49.902373+0000","last_peered":"2024-04-18T22:26:49.902373+0000","last_clean":"2024-04-18T22:26:49.902373+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:49.902373+0000","last_undegraded":"2024-04-18T22:26:49.902373+0000","last_fullsized":"2024-04-18T22:26:49.902373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:49.902160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:49.902160+0000","last_clean_scrub_stamp":"2024-04-18T22:26:49.902160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:02.183017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339935+0000","last_change":"2024-04-18T22:26:34.225372+0000","last_active":"2024-04-18T22:26:38.339935+0000","last_peered":"2024-04-18T22:26:38.339935+0000","last_clean":"2024-04-18T22:26:38.339935+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:26:38.339935+0000","last_undegraded":"2024-04-18T22:26:38.339935+0000","last_fullsized":"2024-04-18T22:26:38.339935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"83'12","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:55.680720+0000","last_change":"2024-04-18T22:26:55.680797+0000","last_active":"2024-04-18T22:26:55.680720+0000","last_peered":"2024-04-18T22:26:55.680720+0000","last_clean":"2024-04-18T22:26:55.680720+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:55.680720+0000","last_undegraded":"2024-04-18T22:26:55.680720+0000","last_fullsized":"2024-04-18T22:26:55.680720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'12","last_scrub_stamp":"2024-04-18T22:26:55.680541+0000","last_deep_scrub":"83'12","last_deep_scrub_stamp":"2024-04-18T22:26:55.680541+0000","last_clean_scrub_stamp":"2024-04-18T22:26:55.680541+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:20:05.389327+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"17'1","reported_seq":117,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.992090+0000","last_change":"2024-04-18T22:26:37.241745+0000","last_active":"2024-04-18T22:26:46.992090+0000","last_peered":"2024-04-18T22:26:46.992090+0000","last_clean":"2024-04-18T22:26:46.992090+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:26:46.992090+0000","last_undegraded":"2024-04-18T22:26:46.992090+0000","last_fullsized":"2024-04-18T22:26:46.992090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209440+0000","last_change":"2024-04-18T22:26:26.058289+0000","last_active":"2024-04-18T22:26:38.209440+0000","last_peered":"2024-04-18T22:26:38.209440+0000","last_clean":"2024-04-18T22:26:38.209440+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:26:38.209440+0000","last_undegraded":"2024-04-18T22:26:38.209440+0000","last_fullsized":"2024-04-18T22:26:38.209440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"83'3","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:56.760117+0000","last_change":"2024-04-18T22:26:56.760178+0000","last_active":"2024-04-18T22:26:56.760117+0000","last_peered":"2024-04-18T22:26:56.760117+0000","last_clean":"2024-04-18T22:26:56.760117+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:56.760117+0000","last_undegraded":"2024-04-18T22:26:56.760117+0000","last_fullsized":"2024-04-18T22:26:56.760117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:56.759936+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:56.759936+0000","last_clean_scrub_stamp":"2024-04-18T22:26:56.759936+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:43.854967+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209596+0000","last_change":"2024-04-18T22:26:26.034657+0000","last_active":"2024-04-18T22:26:38.209596+0000","last_peered":"2024-04-18T22:26:38.209596+0000","last_clean":"2024-04-18T22:26:38.209596+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:26:38.209596+0000","last_undegraded":"2024-04-18T22:26:38.209596+0000","last_fullsized":"2024-04-18T22:26:38.209596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"17'1","reported_seq":117,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.992036+0000","last_change":"2024-04-18T22:26:37.257176+0000","last_active":"2024-04-18T22:26:46.992036+0000","last_peered":"2024-04-18T22:26:46.992036+0000","last_clean":"2024-04-18T22:26:46.992036+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:26:46.992036+0000","last_undegraded":"2024-04-18T22:26:46.992036+0000","last_fullsized":"2024-04-18T22:26:46.992036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.992370+0000","last_change":"2024-04-18T22:26:37.251681+0000","last_active":"2024-04-18T22:26:46.992370+0000","last_peered":"2024-04-18T22:26:46.992370+0000","last_clean":"2024-04-18T22:26:46.992370+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:26:46.992370+0000","last_undegraded":"2024-04-18T22:26:46.992370+0000","last_fullsized":"2024-04-18T22:26:46.992370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:56.724345+0000","last_change":"2024-04-18T22:26:56.724398+0000","last_active":"2024-04-18T22:26:56.724345+0000","last_peered":"2024-04-18T22:26:56.724345+0000","last_clean":"2024-04-18T22:26:56.724345+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:56.724345+0000","last_undegraded":"2024-04-18T22:26:56.724345+0000","last_fullsized":"2024-04-18T22:26:56.724345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:56.724139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:56.724139+0000","last_clean_scrub_stamp":"2024-04-18T22:26:56.724139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:23:42.605818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.064351+0000","last_change":"2024-04-18T22:22:32.143713+0000","last_active":"2024-04-18T22:26:46.064351+0000","last_peered":"2024-04-18T22:26:46.064351+0000","last_clean":"2024-04-18T22:26:46.064351+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:26:46.064351+0000","last_undegraded":"2024-04-18T22:26:46.064351+0000","last_fullsized":"2024-04-18T22:26:46.064351+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"17'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339768+0000","last_change":"2024-04-18T22:26:34.233365+0000","last_active":"2024-04-18T22:26:38.339768+0000","last_peered":"2024-04-18T22:26:38.339768+0000","last_clean":"2024-04-18T22:26:38.339768+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:26:38.339768+0000","last_undegraded":"2024-04-18T22:26:38.339768+0000","last_fullsized":"2024-04-18T22:26:38.339768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:52.713924+0000","last_change":"2024-04-18T22:26:52.713924+0000","last_active":"2024-04-18T22:26:52.713924+0000","last_peered":"2024-04-18T22:26:52.713924+0000","last_clean":"2024-04-18T22:26:52.713924+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:52.713924+0000","last_undegraded":"2024-04-18T22:26:52.713924+0000","last_fullsized":"2024-04-18T22:26:52.713924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:52.713672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:52.713672+0000","last_clean_scrub_stamp":"2024-04-18T22:26:52.713672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:18:16.911539+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.757847+0000","last_change":"2024-04-18T22:27:04.757899+0000","last_active":"2024-04-18T22:27:04.757847+0000","last_peered":"2024-04-18T22:27:04.757847+0000","last_clean":"2024-04-18T22:27:04.757847+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:27:04.757847+0000","last_undegraded":"2024-04-18T22:27:04.757847+0000","last_fullsized":"2024-04-18T22:27:04.757847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.757657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.757657+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.757657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:09:57.845790+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":114,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.992583+0000","last_change":"2024-04-18T22:26:37.257632+0000","last_active":"2024-04-18T22:26:46.992583+0000","last_peered":"2024-04-18T22:26:46.992583+0000","last_clean":"2024-04-18T22:26:46.992583+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:26:46.992583+0000","last_undegraded":"2024-04-18T22:26:46.992583+0000","last_fullsized":"2024-04-18T22:26:46.992583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:48.881120+0000","last_change":"2024-04-18T22:26:48.881171+0000","last_active":"2024-04-18T22:26:48.881120+0000","last_peered":"2024-04-18T22:26:48.881120+0000","last_clean":"2024-04-18T22:26:48.881120+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:48.881120+0000","last_undegraded":"2024-04-18T22:26:48.881120+0000","last_fullsized":"2024-04-18T22:26:48.881120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:48.880922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:48.880922+0000","last_clean_scrub_stamp":"2024-04-18T22:26:48.880922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:03:22.286281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":114,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.992597+0000","last_change":"2024-04-18T22:26:37.244811+0000","last_active":"2024-04-18T22:26:46.992597+0000","last_peered":"2024-04-18T22:26:46.992597+0000","last_clean":"2024-04-18T22:26:46.992597+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:26:46.992597+0000","last_undegraded":"2024-04-18T22:26:46.992597+0000","last_fullsized":"2024-04-18T22:26:46.992597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209791+0000","last_change":"2024-04-18T22:26:26.036291+0000","last_active":"2024-04-18T22:26:38.209791+0000","last_peered":"2024-04-18T22:26:38.209791+0000","last_clean":"2024-04-18T22:26:38.209791+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:26:38.209791+0000","last_undegraded":"2024-04-18T22:26:38.209791+0000","last_fullsized":"2024-04-18T22:26:38.209791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:48.191395+0000","last_change":"2024-04-18T22:26:48.191395+0000","last_active":"2024-04-18T22:26:48.191395+0000","last_peered":"2024-04-18T22:26:48.191395+0000","last_clean":"2024-04-18T22:26:48.191395+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:48.191395+0000","last_undegraded":"2024-04-18T22:26:48.191395+0000","last_fullsized":"2024-04-18T22:26:48.191395+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:48.191250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:48.191250+0000","last_clean_scrub_stamp":"2024-04-18T22:26:48.191250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:32:54.665666+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"83'159","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.848528+0000","last_change":"2024-04-18T22:27:00.848605+0000","last_active":"2024-04-18T22:27:00.848528+0000","last_peered":"2024-04-18T22:27:00.848528+0000","last_clean":"2024-04-18T22:27:00.848528+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:27:00.848528+0000","last_undegraded":"2024-04-18T22:27:00.848528+0000","last_fullsized":"2024-04-18T22:27:00.848528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'159","last_scrub_stamp":"2024-04-18T22:27:00.848340+0000","last_deep_scrub":"83'159","last_deep_scrub_stamp":"2024-04-18T22:27:00.848340+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.848340+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:25:52.754235+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:01.772936+0000","last_change":"2024-04-18T22:27:01.772987+0000","last_active":"2024-04-18T22:27:01.772936+0000","last_peered":"2024-04-18T22:27:01.772936+0000","last_clean":"2024-04-18T22:27:01.772936+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:27:01.772936+0000","last_undegraded":"2024-04-18T22:27:01.772936+0000","last_fullsized":"2024-04-18T22:27:01.772936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:01.772738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:01.772738+0000","last_clean_scrub_stamp":"2024-04-18T22:27:01.772738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:05:02.982896+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.044809+0000","last_change":"2024-04-18T22:26:58.044864+0000","last_active":"2024-04-18T22:26:58.044809+0000","last_peered":"2024-04-18T22:26:58.044809+0000","last_clean":"2024-04-18T22:26:58.044809+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:58.044809+0000","last_undegraded":"2024-04-18T22:26:58.044809+0000","last_fullsized":"2024-04-18T22:26:58.044809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.044612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.044612+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.044612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:11.405080+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:48.745353+0000","last_change":"2024-04-18T22:26:48.745407+0000","last_active":"2024-04-18T22:26:48.745353+0000","last_peered":"2024-04-18T22:26:48.745353+0000","last_clean":"2024-04-18T22:26:48.745353+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:48.745353+0000","last_undegraded":"2024-04-18T22:26:48.745353+0000","last_fullsized":"2024-04-18T22:26:48.745353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:48.745171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:48.745171+0000","last_clean_scrub_stamp":"2024-04-18T22:26:48.745171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:05:58.049454+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.051816+0000","last_change":"2024-04-18T22:26:37.244470+0000","last_active":"2024-04-18T22:26:39.051816+0000","last_peered":"2024-04-18T22:26:39.051816+0000","last_clean":"2024-04-18T22:26:39.051816+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:26:39.051816+0000","last_undegraded":"2024-04-18T22:26:39.051816+0000","last_fullsized":"2024-04-18T22:26:39.051816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.007241+0000","last_change":"2024-04-18T22:27:04.007293+0000","last_active":"2024-04-18T22:27:04.007241+0000","last_peered":"2024-04-18T22:27:04.007241+0000","last_clean":"2024-04-18T22:27:04.007241+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:27:04.007241+0000","last_undegraded":"2024-04-18T22:27:04.007241+0000","last_fullsized":"2024-04-18T22:27:04.007241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.007052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.007052+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.007052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:28:44.608219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211402+0000","last_change":"2024-04-18T22:26:34.196384+0000","last_active":"2024-04-18T22:26:38.211402+0000","last_peered":"2024-04-18T22:26:38.211402+0000","last_clean":"2024-04-18T22:26:38.211402+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:26:38.211402+0000","last_undegraded":"2024-04-18T22:26:38.211402+0000","last_fullsized":"2024-04-18T22:26:38.211402+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:57.052337+0000","last_change":"2024-04-18T22:26:57.052395+0000","last_active":"2024-04-18T22:26:57.052337+0000","last_peered":"2024-04-18T22:26:57.052337+0000","last_clean":"2024-04-18T22:26:57.052337+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:57.052337+0000","last_undegraded":"2024-04-18T22:26:57.052337+0000","last_fullsized":"2024-04-18T22:26:57.052337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:57.052137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:57.052137+0000","last_clean_scrub_stamp":"2024-04-18T22:26:57.052137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:06.647950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.866157+0000","last_change":"2024-04-18T22:26:46.866228+0000","last_active":"2024-04-18T22:26:46.866157+0000","last_peered":"2024-04-18T22:26:46.866157+0000","last_clean":"2024-04-18T22:26:46.866157+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:46.866157+0000","last_undegraded":"2024-04-18T22:26:46.866157+0000","last_fullsized":"2024-04-18T22:26:46.866157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:46.865964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:46.865964+0000","last_clean_scrub_stamp":"2024-04-18T22:26:46.865964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:50:11.432855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340336+0000","last_change":"2024-04-18T22:26:34.225405+0000","last_active":"2024-04-18T22:26:38.340336+0000","last_peered":"2024-04-18T22:26:38.340336+0000","last_clean":"2024-04-18T22:26:38.340336+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:26:38.340336+0000","last_undegraded":"2024-04-18T22:26:38.340336+0000","last_fullsized":"2024-04-18T22:26:38.340336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.051923+0000","last_change":"2024-04-18T22:26:37.241700+0000","last_active":"2024-04-18T22:26:39.051923+0000","last_peered":"2024-04-18T22:26:39.051923+0000","last_clean":"2024-04-18T22:26:39.051923+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:26:39.051923+0000","last_undegraded":"2024-04-18T22:26:39.051923+0000","last_fullsized":"2024-04-18T22:26:39.051923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.752829+0000","last_change":"2024-04-18T22:26:58.752885+0000","last_active":"2024-04-18T22:26:58.752829+0000","last_peered":"2024-04-18T22:26:58.752829+0000","last_clean":"2024-04-18T22:26:58.752829+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:58.752829+0000","last_undegraded":"2024-04-18T22:26:58.752829+0000","last_fullsized":"2024-04-18T22:26:58.752829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.752607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.752607+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.752607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:35:57.342452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.339517+0000","last_change":"2024-04-18T22:26:34.221304+0000","last_active":"2024-04-18T22:26:38.339517+0000","last_peered":"2024-04-18T22:26:38.339517+0000","last_clean":"2024-04-18T22:26:38.339517+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:26:38.339517+0000","last_undegraded":"2024-04-18T22:26:38.339517+0000","last_fullsized":"2024-04-18T22:26:38.339517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.052092+0000","last_change":"2024-04-18T22:26:37.257769+0000","last_active":"2024-04-18T22:26:39.052092+0000","last_peered":"2024-04-18T22:26:39.052092+0000","last_clean":"2024-04-18T22:26:39.052092+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:26:39.052092+0000","last_undegraded":"2024-04-18T22:26:39.052092+0000","last_fullsized":"2024-04-18T22:26:39.052092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:55.063289+0000","last_change":"2024-04-18T22:26:55.063345+0000","last_active":"2024-04-18T22:26:55.063289+0000","last_peered":"2024-04-18T22:26:55.063289+0000","last_clean":"2024-04-18T22:26:55.063289+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:55.063289+0000","last_undegraded":"2024-04-18T22:26:55.063289+0000","last_fullsized":"2024-04-18T22:26:55.063289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:55.063087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:55.063087+0000","last_clean_scrub_stamp":"2024-04-18T22:26:55.063087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:12:15.782434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:51.869133+0000","last_change":"2024-04-18T22:26:51.869187+0000","last_active":"2024-04-18T22:26:51.869133+0000","last_peered":"2024-04-18T22:26:51.869133+0000","last_clean":"2024-04-18T22:26:51.869133+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:51.869133+0000","last_undegraded":"2024-04-18T22:26:51.869133+0000","last_fullsized":"2024-04-18T22:26:51.869133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:51.868951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:51.868951+0000","last_clean_scrub_stamp":"2024-04-18T22:26:51.868951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:16:34.019162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:52.119518+0000","last_change":"2024-04-18T22:26:52.119573+0000","last_active":"2024-04-18T22:26:52.119518+0000","last_peered":"2024-04-18T22:26:52.119518+0000","last_clean":"2024-04-18T22:26:52.119518+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:52.119518+0000","last_undegraded":"2024-04-18T22:26:52.119518+0000","last_fullsized":"2024-04-18T22:26:52.119518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:52.119319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:52.119319+0000","last_clean_scrub_stamp":"2024-04-18T22:26:52.119319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:48.419694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211864+0000","last_change":"2024-04-18T22:26:34.187241+0000","last_active":"2024-04-18T22:26:38.211864+0000","last_peered":"2024-04-18T22:26:38.211864+0000","last_clean":"2024-04-18T22:26:38.211864+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:26:38.211864+0000","last_undegraded":"2024-04-18T22:26:38.211864+0000","last_fullsized":"2024-04-18T22:26:38.211864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:55.801445+0000","last_change":"2024-04-18T22:26:55.801497+0000","last_active":"2024-04-18T22:26:55.801445+0000","last_peered":"2024-04-18T22:26:55.801445+0000","last_clean":"2024-04-18T22:26:55.801445+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:55.801445+0000","last_undegraded":"2024-04-18T22:26:55.801445+0000","last_fullsized":"2024-04-18T22:26:55.801445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:55.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:55.801255+0000","last_clean_scrub_stamp":"2024-04-18T22:26:55.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:40:07.910755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"83'3","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:50.118728+0000","last_change":"2024-04-18T22:26:50.118728+0000","last_active":"2024-04-18T22:26:50.118728+0000","last_peered":"2024-04-18T22:26:50.118728+0000","last_clean":"2024-04-18T22:26:50.118728+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:50.118728+0000","last_undegraded":"2024-04-18T22:26:50.118728+0000","last_fullsized":"2024-04-18T22:26:50.118728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:50.118565+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:50.118565+0000","last_clean_scrub_stamp":"2024-04-18T22:26:50.118565+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:52.966118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:56.030184+0000","last_change":"2024-04-18T22:26:56.030238+0000","last_active":"2024-04-18T22:26:56.030184+0000","last_peered":"2024-04-18T22:26:56.030184+0000","last_clean":"2024-04-18T22:26:56.030184+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:56.030184+0000","last_undegraded":"2024-04-18T22:26:56.030184+0000","last_fullsized":"2024-04-18T22:26:56.030184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:56.029987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:56.029987+0000","last_clean_scrub_stamp":"2024-04-18T22:26:56.029987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:52:23.546855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338179+0000","last_change":"2024-04-18T22:26:34.222501+0000","last_active":"2024-04-18T22:26:38.338179+0000","last_peered":"2024-04-18T22:26:38.338179+0000","last_clean":"2024-04-18T22:26:38.338179+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:26:38.338179+0000","last_undegraded":"2024-04-18T22:26:38.338179+0000","last_fullsized":"2024-04-18T22:26:38.338179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"18'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340710+0000","last_change":"2024-04-18T22:26:34.246405+0000","last_active":"2024-04-18T22:26:38.340710+0000","last_peered":"2024-04-18T22:26:38.340710+0000","last_clean":"2024-04-18T22:26:38.340710+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:26:38.340710+0000","last_undegraded":"2024-04-18T22:26:38.340710+0000","last_fullsized":"2024-04-18T22:26:38.340710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.991927+0000","last_change":"2024-04-18T22:26:37.242330+0000","last_active":"2024-04-18T22:26:46.991927+0000","last_peered":"2024-04-18T22:26:46.991927+0000","last_clean":"2024-04-18T22:26:46.991927+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:26:46.991927+0000","last_undegraded":"2024-04-18T22:26:46.991927+0000","last_fullsized":"2024-04-18T22:26:46.991927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:26.066329+0000","last_clean_scrub_stamp":"2024-04-18T22:22:26.066329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:01.702948+0000","last_change":"2024-04-18T22:27:01.703002+0000","last_active":"2024-04-18T22:27:01.702948+0000","last_peered":"2024-04-18T22:27:01.702948+0000","last_clean":"2024-04-18T22:27:01.702948+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:27:01.702948+0000","last_undegraded":"2024-04-18T22:27:01.702948+0000","last_fullsized":"2024-04-18T22:27:01.702948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:01.702757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:01.702757+0000","last_clean_scrub_stamp":"2024-04-18T22:27:01.702757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:31:51.781627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:49.724701+0000","last_change":"2024-04-18T22:26:49.724754+0000","last_active":"2024-04-18T22:26:49.724701+0000","last_peered":"2024-04-18T22:26:49.724701+0000","last_clean":"2024-04-18T22:26:49.724701+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:49.724701+0000","last_undegraded":"2024-04-18T22:26:49.724701+0000","last_fullsized":"2024-04-18T22:26:49.724701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:49.724511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:49.724511+0000","last_clean_scrub_stamp":"2024-04-18T22:26:49.724511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:28:29.140137+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"17'6","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.241033+0000","last_change":"2024-04-18T22:26:37.245789+0000","last_active":"2024-04-18T22:26:38.241033+0000","last_peered":"2024-04-18T22:26:38.241033+0000","last_clean":"2024-04-18T22:26:38.241033+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:26:38.241033+0000","last_undegraded":"2024-04-18T22:26:38.241033+0000","last_fullsized":"2024-04-18T22:26:38.241033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.990650+0000","last_change":"2024-04-18T22:27:00.990708+0000","last_active":"2024-04-18T22:27:00.990650+0000","last_peered":"2024-04-18T22:27:00.990650+0000","last_clean":"2024-04-18T22:27:00.990650+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:27:00.990650+0000","last_undegraded":"2024-04-18T22:27:00.990650+0000","last_fullsized":"2024-04-18T22:27:00.990650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:00.990461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:00.990461+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.990461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:40.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"83'5","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:47.863158+0000","last_change":"2024-04-18T22:26:47.863223+0000","last_active":"2024-04-18T22:26:47.863158+0000","last_peered":"2024-04-18T22:26:47.863158+0000","last_clean":"2024-04-18T22:26:47.863158+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:47.863158+0000","last_undegraded":"2024-04-18T22:26:47.863158+0000","last_fullsized":"2024-04-18T22:26:47.863158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-04-18T22:26:47.862971+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-04-18T22:26:47.862971+0000","last_clean_scrub_stamp":"2024-04-18T22:26:47.862971+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:01.341015+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:02.786986+0000","last_change":"2024-04-18T22:27:02.787041+0000","last_active":"2024-04-18T22:27:02.786986+0000","last_peered":"2024-04-18T22:27:02.786986+0000","last_clean":"2024-04-18T22:27:02.786986+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:27:02.786986+0000","last_undegraded":"2024-04-18T22:27:02.786986+0000","last_fullsized":"2024-04-18T22:27:02.786986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:02.786795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:02.786795+0000","last_clean_scrub_stamp":"2024-04-18T22:27:02.786795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:59.056804+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.338533+0000","last_change":"2024-04-18T22:22:32.143683+0000","last_active":"2024-04-18T22:26:38.338533+0000","last_peered":"2024-04-18T22:26:38.338533+0000","last_clean":"2024-04-18T22:26:38.338533+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:26:38.338533+0000","last_undegraded":"2024-04-18T22:26:38.338533+0000","last_fullsized":"2024-04-18T22:26:38.338533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340869+0000","last_change":"2024-04-18T22:22:31.141168+0000","last_active":"2024-04-18T22:26:38.340869+0000","last_peered":"2024-04-18T22:26:38.340869+0000","last_clean":"2024-04-18T22:26:38.340869+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:26:38.340869+0000","last_undegraded":"2024-04-18T22:26:38.340869+0000","last_fullsized":"2024-04-18T22:26:38.340869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:47.051716+0000","last_change":"2024-04-18T22:26:47.051774+0000","last_active":"2024-04-18T22:26:47.051716+0000","last_peered":"2024-04-18T22:26:47.051716+0000","last_clean":"2024-04-18T22:26:47.051716+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:47.051716+0000","last_undegraded":"2024-04-18T22:26:47.051716+0000","last_fullsized":"2024-04-18T22:26:47.051716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:47.051489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:47.051489+0000","last_clean_scrub_stamp":"2024-04-18T22:26:47.051489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:53:21.437741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:50.993560+0000","last_change":"2024-04-18T22:26:50.993683+0000","last_active":"2024-04-18T22:26:50.993560+0000","last_peered":"2024-04-18T22:26:50.993560+0000","last_clean":"2024-04-18T22:26:50.993560+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:50.993560+0000","last_undegraded":"2024-04-18T22:26:50.993560+0000","last_fullsized":"2024-04-18T22:26:50.993560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:50.993315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:50.993315+0000","last_clean_scrub_stamp":"2024-04-18T22:26:50.993315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:24.120390+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:54.071697+0000","last_change":"2024-04-18T22:26:54.071750+0000","last_active":"2024-04-18T22:26:54.071697+0000","last_peered":"2024-04-18T22:26:54.071697+0000","last_clean":"2024-04-18T22:26:54.071697+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:54.071697+0000","last_undegraded":"2024-04-18T22:26:54.071697+0000","last_fullsized":"2024-04-18T22:26:54.071697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:54.071502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:54.071502+0000","last_clean_scrub_stamp":"2024-04-18T22:26:54.071502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:57:11.686559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:03.050790+0000","last_change":"2024-04-18T22:27:03.050845+0000","last_active":"2024-04-18T22:27:03.050790+0000","last_peered":"2024-04-18T22:27:03.050790+0000","last_clean":"2024-04-18T22:27:03.050790+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:27:03.050790+0000","last_undegraded":"2024-04-18T22:27:03.050790+0000","last_fullsized":"2024-04-18T22:27:03.050790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:03.050604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:03.050604+0000","last_clean_scrub_stamp":"2024-04-18T22:27:03.050604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:20:12.759390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:02.003535+0000","last_change":"2024-04-18T22:27:02.003589+0000","last_active":"2024-04-18T22:27:02.003535+0000","last_peered":"2024-04-18T22:27:02.003535+0000","last_clean":"2024-04-18T22:27:02.003535+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:27:02.003535+0000","last_undegraded":"2024-04-18T22:27:02.003535+0000","last_fullsized":"2024-04-18T22:27:02.003535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:02.003343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:02.003343+0000","last_clean_scrub_stamp":"2024-04-18T22:27:02.003343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:06.603570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:02.723432+0000","last_change":"2024-04-18T22:27:02.723485+0000","last_active":"2024-04-18T22:27:02.723432+0000","last_peered":"2024-04-18T22:27:02.723432+0000","last_clean":"2024-04-18T22:27:02.723432+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:27:02.723432+0000","last_undegraded":"2024-04-18T22:27:02.723432+0000","last_fullsized":"2024-04-18T22:27:02.723432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:02.723244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:02.723244+0000","last_clean_scrub_stamp":"2024-04-18T22:27:02.723244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:31:34.496835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.210118+0000","last_change":"2024-04-18T22:26:26.035577+0000","last_active":"2024-04-18T22:26:38.210118+0000","last_peered":"2024-04-18T22:26:38.210118+0000","last_clean":"2024-04-18T22:26:38.210118+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:26:38.210118+0000","last_undegraded":"2024-04-18T22:26:38.210118+0000","last_fullsized":"2024-04-18T22:26:38.210118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.211177+0000","last_change":"2024-04-18T22:26:34.196548+0000","last_active":"2024-04-18T22:26:38.211177+0000","last_peered":"2024-04-18T22:26:38.211177+0000","last_clean":"2024-04-18T22:26:38.211177+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:26:38.211177+0000","last_undegraded":"2024-04-18T22:26:38.211177+0000","last_fullsized":"2024-04-18T22:26:38.211177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:59.769787+0000","last_change":"2024-04-18T22:26:59.769850+0000","last_active":"2024-04-18T22:26:59.769787+0000","last_peered":"2024-04-18T22:26:59.769787+0000","last_clean":"2024-04-18T22:26:59.769787+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:59.769787+0000","last_undegraded":"2024-04-18T22:26:59.769787+0000","last_fullsized":"2024-04-18T22:26:59.769787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'1","last_scrub_stamp":"2024-04-18T22:26:59.769591+0000","last_deep_scrub":"17'1","last_deep_scrub_stamp":"2024-04-18T22:26:59.769591+0000","last_clean_scrub_stamp":"2024-04-18T22:26:59.769591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:24:05.029508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:51.800244+0000","last_change":"2024-04-18T22:26:51.800299+0000","last_active":"2024-04-18T22:26:51.800244+0000","last_peered":"2024-04-18T22:26:51.800244+0000","last_clean":"2024-04-18T22:26:51.800244+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:51.800244+0000","last_undegraded":"2024-04-18T22:26:51.800244+0000","last_fullsized":"2024-04-18T22:26:51.800244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:51.800067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:51.800067+0000","last_clean_scrub_stamp":"2024-04-18T22:26:51.800067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:53.488490+0000","scrub_duration":95,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"83'3","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:47.730120+0000","last_change":"2024-04-18T22:26:47.730181+0000","last_active":"2024-04-18T22:26:47.730120+0000","last_peered":"2024-04-18T22:26:47.730120+0000","last_clean":"2024-04-18T22:26:47.730120+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:47.730120+0000","last_undegraded":"2024-04-18T22:26:47.730120+0000","last_fullsized":"2024-04-18T22:26:47.730120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:47.729935+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:47.729935+0000","last_clean_scrub_stamp":"2024-04-18T22:26:47.729935+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:12:08.449282+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:39.051887+0000","last_change":"2024-04-18T22:26:37.242414+0000","last_active":"2024-04-18T22:26:39.051887+0000","last_peered":"2024-04-18T22:26:39.051887+0000","last_clean":"2024-04-18T22:26:39.051887+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:26:39.051887+0000","last_undegraded":"2024-04-18T22:26:39.051887+0000","last_fullsized":"2024-04-18T22:26:39.051887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"83'3","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:54.842969+0000","last_change":"2024-04-18T22:26:54.843035+0000","last_active":"2024-04-18T22:26:54.842969+0000","last_peered":"2024-04-18T22:26:54.842969+0000","last_clean":"2024-04-18T22:26:54.842969+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:54.842969+0000","last_undegraded":"2024-04-18T22:26:54.842969+0000","last_fullsized":"2024-04-18T22:26:54.842969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:54.842768+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:54.842768+0000","last_clean_scrub_stamp":"2024-04-18T22:26:54.842768+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:31:45.020434+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.209932+0000","last_change":"2024-04-18T22:26:26.034893+0000","last_active":"2024-04-18T22:26:38.209932+0000","last_peered":"2024-04-18T22:26:38.209932+0000","last_clean":"2024-04-18T22:26:38.209932+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:26:38.209932+0000","last_undegraded":"2024-04-18T22:26:38.209932+0000","last_fullsized":"2024-04-18T22:26:38.209932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":162,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:53.082011+0000","last_change":"2024-04-18T22:26:53.082081+0000","last_active":"2024-04-18T22:26:53.082011+0000","last_peered":"2024-04-18T22:26:53.082011+0000","last_clean":"2024-04-18T22:26:53.082011+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:53.082011+0000","last_undegraded":"2024-04-18T22:26:53.082011+0000","last_fullsized":"2024-04-18T22:26:53.082011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'1","last_scrub_stamp":"2024-04-18T22:26:53.081817+0000","last_deep_scrub":"17'1","last_deep_scrub_stamp":"2024-04-18T22:26:53.081817+0000","last_clean_scrub_stamp":"2024-04-18T22:26:53.081817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:27.621584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":92,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.239344+0000","last_change":"2024-04-18T22:26:37.245625+0000","last_active":"2024-04-18T22:26:38.239344+0000","last_peered":"2024-04-18T22:26:38.239344+0000","last_clean":"2024-04-18T22:26:38.239344+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:26:38.239344+0000","last_undegraded":"2024-04-18T22:26:38.239344+0000","last_fullsized":"2024-04-18T22:26:38.239344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:31.093710+0000","last_clean_scrub_stamp":"2024-04-18T22:22:31.093710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"17'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:38.340326+0000","last_change":"2024-04-18T22:26:34.256089+0000","last_active":"2024-04-18T22:26:38.340326+0000","last_peered":"2024-04-18T22:26:38.340326+0000","last_clean":"2024-04-18T22:26:38.340326+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:26:38.340326+0000","last_undegraded":"2024-04-18T22:26:38.340326+0000","last_fullsized":"2024-04-18T22:26:38.340326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:22:30.083100+0000","last_clean_scrub_stamp":"2024-04-18T22:22:30.083100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738430,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293160,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078680,"statfs":{"total":96636764160,"available":96336568320,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547210,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41888,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329952,"statfs":{"total":96636764160,"available":96593870848,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334744,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037096,"statfs":{"total":96636764160,"available":96293986304,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:27:07.682 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.683 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.684 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.685 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.686 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-18T22:22:26.066329+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=26, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.687 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.688 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-18T22:22:31.093710+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=31, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.689 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-18T22:22:30.083100+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=22, tm_sec=30, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=22, tm_min=26, tm_sec=44, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T22:27:07.689 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-18T22:27:27.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T22:27:28.015 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:27:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-18T22:27:28.030 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-04-18T22:27:26.964855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":669792,"kb_used_data":2888,"kb_used_omap":8,"kb_used_meta":666871,"kb_avail":282445728,"statfs":{"total":289910292480,"available":289224425472,"internally_reserved":0,"allocated":2957312,"data_stored":2331904,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9202,"internal_metadata":682875918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004349"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:19.851725+0000","last_change":"2024-04-18T22:27:19.851781+0000","last_active":"2024-04-18T22:27:19.851725+0000","last_peered":"2024-04-18T22:27:19.851725+0000","last_clean":"2024-04-18T22:27:19.851725+0000","last_became_active":"2024-04-18T22:26:34.195917+0000","last_became_peered":"2024-04-18T22:26:34.195917+0000","last_unstale":"2024-04-18T22:27:19.851725+0000","last_undegraded":"2024-04-18T22:27:19.851725+0000","last_fullsized":"2024-04-18T22:27:19.851725+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:19.851469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:19.851469+0000","last_clean_scrub_stamp":"2024-04-18T22:27:19.851469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:31:26.015118+0000","scrub_duration":139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:03.702263+0000","last_change":"2024-04-18T22:27:03.702317+0000","last_active":"2024-04-18T22:27:03.702263+0000","last_peered":"2024-04-18T22:27:03.702263+0000","last_clean":"2024-04-18T22:27:03.702263+0000","last_became_active":"2024-04-18T22:26:26.035504+0000","last_became_peered":"2024-04-18T22:26:26.035504+0000","last_unstale":"2024-04-18T22:27:03.702263+0000","last_undegraded":"2024-04-18T22:27:03.702263+0000","last_fullsized":"2024-04-18T22:27:03.702263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:03.702062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:03.702062+0000","last_clean_scrub_stamp":"2024-04-18T22:27:03.702062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:16:32.180267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:57.764363+0000","last_change":"2024-04-18T22:26:57.764480+0000","last_active":"2024-04-18T22:26:57.764363+0000","last_peered":"2024-04-18T22:26:57.764363+0000","last_clean":"2024-04-18T22:26:57.764363+0000","last_became_active":"2024-04-18T22:26:37.245625+0000","last_became_peered":"2024-04-18T22:26:37.245625+0000","last_unstale":"2024-04-18T22:26:57.764363+0000","last_undegraded":"2024-04-18T22:26:57.764363+0000","last_fullsized":"2024-04-18T22:26:57.764363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:57.764113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:57.764113+0000","last_clean_scrub_stamp":"2024-04-18T22:26:57.764113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:21:34.665656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:20.774203+0000","last_change":"2024-04-18T22:27:20.774256+0000","last_active":"2024-04-18T22:27:20.774203+0000","last_peered":"2024-04-18T22:27:20.774203+0000","last_clean":"2024-04-18T22:27:20.774203+0000","last_became_active":"2024-04-18T22:26:37.254109+0000","last_became_peered":"2024-04-18T22:26:37.254109+0000","last_unstale":"2024-04-18T22:27:20.774203+0000","last_undegraded":"2024-04-18T22:27:20.774203+0000","last_fullsized":"2024-04-18T22:27:20.774203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:20.774017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:20.774017+0000","last_clean_scrub_stamp":"2024-04-18T22:27:20.774017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:32:08.147272+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.755214+0000","last_change":"2024-04-18T22:27:04.755269+0000","last_active":"2024-04-18T22:27:04.755214+0000","last_peered":"2024-04-18T22:27:04.755214+0000","last_clean":"2024-04-18T22:27:04.755214+0000","last_became_active":"2024-04-18T22:26:26.033853+0000","last_became_peered":"2024-04-18T22:26:26.033853+0000","last_unstale":"2024-04-18T22:27:04.755214+0000","last_undegraded":"2024-04-18T22:27:04.755214+0000","last_fullsized":"2024-04-18T22:27:04.755214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.755032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.755032+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.755032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:38:17.368614+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:18.735602+0000","last_change":"2024-04-18T22:27:18.735705+0000","last_active":"2024-04-18T22:27:18.735602+0000","last_peered":"2024-04-18T22:27:18.735602+0000","last_clean":"2024-04-18T22:27:18.735602+0000","last_became_active":"2024-04-18T22:26:37.253148+0000","last_became_peered":"2024-04-18T22:26:37.253148+0000","last_unstale":"2024-04-18T22:27:18.735602+0000","last_undegraded":"2024-04-18T22:27:18.735602+0000","last_fullsized":"2024-04-18T22:27:18.735602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:18.735345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:18.735345+0000","last_clean_scrub_stamp":"2024-04-18T22:27:18.735345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:34:52.307227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:10.734549+0000","last_change":"2024-04-18T22:27:10.734607+0000","last_active":"2024-04-18T22:27:10.734549+0000","last_peered":"2024-04-18T22:27:10.734549+0000","last_clean":"2024-04-18T22:27:10.734549+0000","last_became_active":"2024-04-18T22:26:37.234055+0000","last_became_peered":"2024-04-18T22:26:37.234055+0000","last_unstale":"2024-04-18T22:27:10.734549+0000","last_undegraded":"2024-04-18T22:27:10.734549+0000","last_fullsized":"2024-04-18T22:27:10.734549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:10.734357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:10.734357+0000","last_clean_scrub_stamp":"2024-04-18T22:27:10.734357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:08:43.826021+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:03.794704+0000","last_change":"2024-04-18T22:27:03.794755+0000","last_active":"2024-04-18T22:27:03.794704+0000","last_peered":"2024-04-18T22:27:03.794704+0000","last_clean":"2024-04-18T22:27:03.794704+0000","last_became_active":"2024-04-18T22:26:37.249938+0000","last_became_peered":"2024-04-18T22:26:37.249938+0000","last_unstale":"2024-04-18T22:27:03.794704+0000","last_undegraded":"2024-04-18T22:27:03.794704+0000","last_fullsized":"2024-04-18T22:27:03.794704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:03.794514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:03.794514+0000","last_clean_scrub_stamp":"2024-04-18T22:27:03.794514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:59:18.073591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:51.223538+0000","last_change":"2024-04-18T22:26:51.223538+0000","last_active":"2024-04-18T22:26:51.223538+0000","last_peered":"2024-04-18T22:26:51.223538+0000","last_clean":"2024-04-18T22:26:51.223538+0000","last_became_active":"2024-04-18T22:26:26.067890+0000","last_became_peered":"2024-04-18T22:26:26.067890+0000","last_unstale":"2024-04-18T22:26:51.223538+0000","last_undegraded":"2024-04-18T22:26:51.223538+0000","last_fullsized":"2024-04-18T22:26:51.223538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:51.223340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:51.223340+0000","last_clean_scrub_stamp":"2024-04-18T22:26:51.223340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:30:27.396252+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:17.731330+0000","last_change":"2024-04-18T22:27:17.731388+0000","last_active":"2024-04-18T22:27:17.731330+0000","last_peered":"2024-04-18T22:27:17.731330+0000","last_clean":"2024-04-18T22:27:17.731330+0000","last_became_active":"2024-04-18T22:26:34.195641+0000","last_became_peered":"2024-04-18T22:26:34.195641+0000","last_unstale":"2024-04-18T22:27:17.731330+0000","last_undegraded":"2024-04-18T22:27:17.731330+0000","last_fullsized":"2024-04-18T22:27:17.731330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:17.731096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:17.731096+0000","last_clean_scrub_stamp":"2024-04-18T22:27:17.731096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:59:50.684329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:53.711861+0000","last_change":"2024-04-18T22:26:53.711861+0000","last_active":"2024-04-18T22:26:53.711861+0000","last_peered":"2024-04-18T22:26:53.711861+0000","last_clean":"2024-04-18T22:26:53.711861+0000","last_became_active":"2024-04-18T22:26:37.245568+0000","last_became_peered":"2024-04-18T22:26:37.245568+0000","last_unstale":"2024-04-18T22:26:53.711861+0000","last_undegraded":"2024-04-18T22:26:53.711861+0000","last_fullsized":"2024-04-18T22:26:53.711861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:53.711620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:53.711620+0000","last_clean_scrub_stamp":"2024-04-18T22:26:53.711620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:29:43.095832+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.774267+0000","last_change":"2024-04-18T22:26:58.774324+0000","last_active":"2024-04-18T22:26:58.774267+0000","last_peered":"2024-04-18T22:26:58.774267+0000","last_clean":"2024-04-18T22:26:58.774267+0000","last_became_active":"2024-04-18T22:26:26.056377+0000","last_became_peered":"2024-04-18T22:26:26.056377+0000","last_unstale":"2024-04-18T22:26:58.774267+0000","last_undegraded":"2024-04-18T22:26:58.774267+0000","last_fullsized":"2024-04-18T22:26:58.774267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.774079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.774079+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.774079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:50:35.726437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:13.729891+0000","last_change":"2024-04-18T22:27:13.729943+0000","last_active":"2024-04-18T22:27:13.729891+0000","last_peered":"2024-04-18T22:27:13.729891+0000","last_clean":"2024-04-18T22:27:13.729891+0000","last_became_active":"2024-04-18T22:26:34.186111+0000","last_became_peered":"2024-04-18T22:26:34.186111+0000","last_unstale":"2024-04-18T22:27:13.729891+0000","last_undegraded":"2024-04-18T22:27:13.729891+0000","last_fullsized":"2024-04-18T22:27:13.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:13.729695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:13.729695+0000","last_clean_scrub_stamp":"2024-04-18T22:27:13.729695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:27:03.775203+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:50.685388+0000","last_change":"2024-04-18T22:26:50.685388+0000","last_active":"2024-04-18T22:26:50.685388+0000","last_peered":"2024-04-18T22:26:50.685388+0000","last_clean":"2024-04-18T22:26:50.685388+0000","last_became_active":"2024-04-18T22:26:37.250821+0000","last_became_peered":"2024-04-18T22:26:37.250821+0000","last_unstale":"2024-04-18T22:26:50.685388+0000","last_undegraded":"2024-04-18T22:26:50.685388+0000","last_fullsized":"2024-04-18T22:26:50.685388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:50.685135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:50.685135+0000","last_clean_scrub_stamp":"2024-04-18T22:26:50.685135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:01:58.721958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:09.730930+0000","last_change":"2024-04-18T22:27:09.730930+0000","last_active":"2024-04-18T22:27:09.730930+0000","last_peered":"2024-04-18T22:27:09.730930+0000","last_clean":"2024-04-18T22:27:09.730930+0000","last_became_active":"2024-04-18T22:26:37.230217+0000","last_became_peered":"2024-04-18T22:26:37.230217+0000","last_unstale":"2024-04-18T22:27:09.730930+0000","last_undegraded":"2024-04-18T22:27:09.730930+0000","last_fullsized":"2024-04-18T22:27:09.730930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:09.730688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:09.730688+0000","last_clean_scrub_stamp":"2024-04-18T22:27:09.730688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:23.306464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:05.784546+0000","last_change":"2024-04-18T22:27:05.784599+0000","last_active":"2024-04-18T22:27:05.784546+0000","last_peered":"2024-04-18T22:27:05.784546+0000","last_clean":"2024-04-18T22:27:05.784546+0000","last_became_active":"2024-04-18T22:26:37.248996+0000","last_became_peered":"2024-04-18T22:26:37.248996+0000","last_unstale":"2024-04-18T22:27:05.784546+0000","last_undegraded":"2024-04-18T22:27:05.784546+0000","last_fullsized":"2024-04-18T22:27:05.784546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:05.784352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:05.784352+0000","last_clean_scrub_stamp":"2024-04-18T22:27:05.784352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:32:05.226425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:49.193412+0000","last_change":"2024-04-18T22:26:49.193412+0000","last_active":"2024-04-18T22:26:49.193412+0000","last_peered":"2024-04-18T22:26:49.193412+0000","last_clean":"2024-04-18T22:26:49.193412+0000","last_became_active":"2024-04-18T22:26:26.069218+0000","last_became_peered":"2024-04-18T22:26:26.069218+0000","last_unstale":"2024-04-18T22:26:49.193412+0000","last_undegraded":"2024-04-18T22:26:49.193412+0000","last_fullsized":"2024-04-18T22:26:49.193412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:49.193291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:49.193291+0000","last_clean_scrub_stamp":"2024-04-18T22:26:49.193291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:56:57.689686+0000","scrub_duration":119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:10.754684+0000","last_change":"2024-04-18T22:27:10.754737+0000","last_active":"2024-04-18T22:27:10.754684+0000","last_peered":"2024-04-18T22:27:10.754684+0000","last_clean":"2024-04-18T22:27:10.754684+0000","last_became_active":"2024-04-18T22:26:26.034711+0000","last_became_peered":"2024-04-18T22:26:26.034711+0000","last_unstale":"2024-04-18T22:27:10.754684+0000","last_undegraded":"2024-04-18T22:27:10.754684+0000","last_fullsized":"2024-04-18T22:27:10.754684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:10.754488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:10.754488+0000","last_clean_scrub_stamp":"2024-04-18T22:27:10.754488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:08:54.939114+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:14.243539+0000","last_change":"2024-04-18T22:27:14.243622+0000","last_active":"2024-04-18T22:27:14.243539+0000","last_peered":"2024-04-18T22:27:14.243539+0000","last_clean":"2024-04-18T22:27:14.243539+0000","last_became_active":"2024-04-18T22:26:34.224125+0000","last_became_peered":"2024-04-18T22:26:34.224125+0000","last_unstale":"2024-04-18T22:27:14.243539+0000","last_undegraded":"2024-04-18T22:27:14.243539+0000","last_fullsized":"2024-04-18T22:27:14.243539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:14.243301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:14.243301+0000","last_clean_scrub_stamp":"2024-04-18T22:27:14.243301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:23:37.883221+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:19.737963+0000","last_change":"2024-04-18T22:27:19.738015+0000","last_active":"2024-04-18T22:27:19.737963+0000","last_peered":"2024-04-18T22:27:19.737963+0000","last_clean":"2024-04-18T22:27:19.737963+0000","last_became_active":"2024-04-18T22:26:37.253174+0000","last_became_peered":"2024-04-18T22:26:37.253174+0000","last_unstale":"2024-04-18T22:27:19.737963+0000","last_undegraded":"2024-04-18T22:27:19.737963+0000","last_fullsized":"2024-04-18T22:27:19.737963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:19.737769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:19.737769+0000","last_clean_scrub_stamp":"2024-04-18T22:27:19.737769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:00:32.958980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:54.694230+0000","last_change":"2024-04-18T22:26:54.694275+0000","last_active":"2024-04-18T22:26:54.694230+0000","last_peered":"2024-04-18T22:26:54.694230+0000","last_clean":"2024-04-18T22:26:54.694230+0000","last_became_active":"2024-04-18T22:26:37.244916+0000","last_became_peered":"2024-04-18T22:26:37.244916+0000","last_unstale":"2024-04-18T22:26:54.694230+0000","last_undegraded":"2024-04-18T22:26:54.694230+0000","last_fullsized":"2024-04-18T22:26:54.694230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:54.694060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:54.694060+0000","last_clean_scrub_stamp":"2024-04-18T22:26:54.694060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:41:59.294287+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:57.788683+0000","last_change":"2024-04-18T22:26:57.788736+0000","last_active":"2024-04-18T22:26:57.788683+0000","last_peered":"2024-04-18T22:26:57.788683+0000","last_clean":"2024-04-18T22:26:57.788683+0000","last_became_active":"2024-04-18T22:26:26.048558+0000","last_became_peered":"2024-04-18T22:26:26.048558+0000","last_unstale":"2024-04-18T22:26:57.788683+0000","last_undegraded":"2024-04-18T22:26:57.788683+0000","last_fullsized":"2024-04-18T22:26:57.788683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:57.788490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:57.788490+0000","last_clean_scrub_stamp":"2024-04-18T22:26:57.788490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:17:54.948384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:09.070288+0000","last_change":"2024-04-18T22:27:09.070341+0000","last_active":"2024-04-18T22:27:09.070288+0000","last_peered":"2024-04-18T22:27:09.070288+0000","last_clean":"2024-04-18T22:27:09.070288+0000","last_became_active":"2024-04-18T22:22:32.142626+0000","last_became_peered":"2024-04-18T22:22:32.142626+0000","last_unstale":"2024-04-18T22:27:09.070288+0000","last_undegraded":"2024-04-18T22:27:09.070288+0000","last_fullsized":"2024-04-18T22:27:09.070288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:09.070101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:09.070101+0000","last_clean_scrub_stamp":"2024-04-18T22:27:09.070101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:34:53.834804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:09.755479+0000","last_change":"2024-04-18T22:27:09.755529+0000","last_active":"2024-04-18T22:27:09.755479+0000","last_peered":"2024-04-18T22:27:09.755479+0000","last_clean":"2024-04-18T22:27:09.755479+0000","last_became_active":"2024-04-18T22:26:26.034042+0000","last_became_peered":"2024-04-18T22:26:26.034042+0000","last_unstale":"2024-04-18T22:27:09.755479+0000","last_undegraded":"2024-04-18T22:27:09.755479+0000","last_fullsized":"2024-04-18T22:27:09.755479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:09.755283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:09.755283+0000","last_clean_scrub_stamp":"2024-04-18T22:27:09.755283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:17:56.780035+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:52.934833+0000","last_change":"2024-04-18T22:26:52.934890+0000","last_active":"2024-04-18T22:26:52.934833+0000","last_peered":"2024-04-18T22:26:52.934833+0000","last_clean":"2024-04-18T22:26:52.934833+0000","last_became_active":"2024-04-18T22:26:26.052950+0000","last_became_peered":"2024-04-18T22:26:26.052950+0000","last_unstale":"2024-04-18T22:26:52.934833+0000","last_undegraded":"2024-04-18T22:26:52.934833+0000","last_fullsized":"2024-04-18T22:26:52.934833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:52.934621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:52.934621+0000","last_clean_scrub_stamp":"2024-04-18T22:26:52.934621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:35:31.513357+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:11.142429+0000","last_change":"2024-04-18T22:27:11.142487+0000","last_active":"2024-04-18T22:27:11.142429+0000","last_peered":"2024-04-18T22:27:11.142429+0000","last_clean":"2024-04-18T22:27:11.142429+0000","last_became_active":"2024-04-18T22:22:32.143109+0000","last_became_peered":"2024-04-18T22:22:32.143109+0000","last_unstale":"2024-04-18T22:27:11.142429+0000","last_undegraded":"2024-04-18T22:27:11.142429+0000","last_fullsized":"2024-04-18T22:27:11.142429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:11.142238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:11.142238+0000","last_clean_scrub_stamp":"2024-04-18T22:27:11.142238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:07:20.772715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:21.800374+0000","last_change":"2024-04-18T22:27:21.800427+0000","last_active":"2024-04-18T22:27:21.800374+0000","last_peered":"2024-04-18T22:27:21.800374+0000","last_clean":"2024-04-18T22:27:21.800374+0000","last_became_active":"2024-04-18T22:26:37.253504+0000","last_became_peered":"2024-04-18T22:26:37.253504+0000","last_unstale":"2024-04-18T22:27:21.800374+0000","last_undegraded":"2024-04-18T22:27:21.800374+0000","last_fullsized":"2024-04-18T22:27:21.800374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:21.800158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:21.800158+0000","last_clean_scrub_stamp":"2024-04-18T22:27:21.800158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:50:24.479264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.741215+0000","last_change":"2024-04-18T22:27:00.741353+0000","last_active":"2024-04-18T22:27:00.741215+0000","last_peered":"2024-04-18T22:27:00.741215+0000","last_clean":"2024-04-18T22:27:00.741215+0000","last_became_active":"2024-04-18T22:26:26.055941+0000","last_became_peered":"2024-04-18T22:26:26.055941+0000","last_unstale":"2024-04-18T22:27:00.741215+0000","last_undegraded":"2024-04-18T22:27:00.741215+0000","last_fullsized":"2024-04-18T22:27:00.741215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:00.740983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:00.740983+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.740983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:27:43.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.018418+0000","last_change":"2024-04-18T22:27:00.018443+0000","last_active":"2024-04-18T22:27:00.018418+0000","last_peered":"2024-04-18T22:27:00.018418+0000","last_clean":"2024-04-18T22:27:00.018418+0000","last_became_active":"2024-04-18T22:26:34.215133+0000","last_became_peered":"2024-04-18T22:26:34.215133+0000","last_unstale":"2024-04-18T22:27:00.018418+0000","last_undegraded":"2024-04-18T22:27:00.018418+0000","last_fullsized":"2024-04-18T22:27:00.018418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:00.018318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:00.018318+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.018318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:23:42.928114+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:53.876074+0000","last_change":"2024-04-18T22:26:53.876130+0000","last_active":"2024-04-18T22:26:53.876074+0000","last_peered":"2024-04-18T22:26:53.876074+0000","last_clean":"2024-04-18T22:26:53.876074+0000","last_became_active":"2024-04-18T22:26:26.048945+0000","last_became_peered":"2024-04-18T22:26:26.048945+0000","last_unstale":"2024-04-18T22:26:53.876074+0000","last_undegraded":"2024-04-18T22:26:53.876074+0000","last_fullsized":"2024-04-18T22:26:53.876074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:53.875887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:53.875887+0000","last_clean_scrub_stamp":"2024-04-18T22:26:53.875887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:17:28.614068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.995800+0000","last_change":"2024-04-18T22:26:58.995856+0000","last_active":"2024-04-18T22:26:58.995800+0000","last_peered":"2024-04-18T22:26:58.995800+0000","last_clean":"2024-04-18T22:26:58.995800+0000","last_became_active":"2024-04-18T22:26:26.059241+0000","last_became_peered":"2024-04-18T22:26:26.059241+0000","last_unstale":"2024-04-18T22:26:58.995800+0000","last_undegraded":"2024-04-18T22:26:58.995800+0000","last_fullsized":"2024-04-18T22:26:58.995800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.995601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.995601+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.995601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:51:48.807197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:59.762552+0000","last_change":"2024-04-18T22:26:59.762603+0000","last_active":"2024-04-18T22:26:59.762552+0000","last_peered":"2024-04-18T22:26:59.762552+0000","last_clean":"2024-04-18T22:26:59.762552+0000","last_became_active":"2024-04-18T22:26:37.248186+0000","last_became_peered":"2024-04-18T22:26:37.248186+0000","last_unstale":"2024-04-18T22:26:59.762552+0000","last_undegraded":"2024-04-18T22:26:59.762552+0000","last_fullsized":"2024-04-18T22:26:59.762552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:59.762355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:59.762355+0000","last_clean_scrub_stamp":"2024-04-18T22:26:59.762355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:46:46.389631+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:15.754183+0000","last_change":"2024-04-18T22:27:15.754234+0000","last_active":"2024-04-18T22:27:15.754183+0000","last_peered":"2024-04-18T22:27:15.754183+0000","last_clean":"2024-04-18T22:27:15.754183+0000","last_became_active":"2024-04-18T22:26:34.194811+0000","last_became_peered":"2024-04-18T22:26:34.194811+0000","last_unstale":"2024-04-18T22:27:15.754183+0000","last_undegraded":"2024-04-18T22:27:15.754183+0000","last_fullsized":"2024-04-18T22:27:15.754183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:15.753984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:15.753984+0000","last_clean_scrub_stamp":"2024-04-18T22:27:15.753984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:33:07.488542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"83'4","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:17.724886+0000","last_change":"2024-04-18T22:27:17.724951+0000","last_active":"2024-04-18T22:27:17.724886+0000","last_peered":"2024-04-18T22:27:17.724886+0000","last_clean":"2024-04-18T22:27:17.724886+0000","last_became_active":"2024-04-18T22:26:37.247969+0000","last_became_peered":"2024-04-18T22:26:37.247969+0000","last_unstale":"2024-04-18T22:27:17.724886+0000","last_undegraded":"2024-04-18T22:27:17.724886+0000","last_fullsized":"2024-04-18T22:27:17.724886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-04-18T22:27:17.724671+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-04-18T22:27:17.724671+0000","last_clean_scrub_stamp":"2024-04-18T22:27:17.724671+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:27:28.980633+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:05.793151+0000","last_change":"2024-04-18T22:27:05.793203+0000","last_active":"2024-04-18T22:27:05.793151+0000","last_peered":"2024-04-18T22:27:05.793151+0000","last_clean":"2024-04-18T22:27:05.793151+0000","last_became_active":"2024-04-18T22:26:26.033960+0000","last_became_peered":"2024-04-18T22:26:26.033960+0000","last_unstale":"2024-04-18T22:27:05.793151+0000","last_undegraded":"2024-04-18T22:27:05.793151+0000","last_fullsized":"2024-04-18T22:27:05.793151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:05.792967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:05.792967+0000","last_clean_scrub_stamp":"2024-04-18T22:27:05.792967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:32:09.458905+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.982016+0000","last_change":"2024-04-18T22:27:04.982016+0000","last_active":"2024-04-18T22:27:04.982016+0000","last_peered":"2024-04-18T22:27:04.982016+0000","last_clean":"2024-04-18T22:27:04.982016+0000","last_became_active":"2024-04-18T22:22:31.140176+0000","last_became_peered":"2024-04-18T22:22:31.140176+0000","last_unstale":"2024-04-18T22:27:04.982016+0000","last_undegraded":"2024-04-18T22:27:04.982016+0000","last_fullsized":"2024-04-18T22:27:04.982016+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.981759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.981759+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.981759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:45:49.051876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:10.100210+0000","last_change":"2024-04-18T22:27:10.100210+0000","last_active":"2024-04-18T22:27:10.100210+0000","last_peered":"2024-04-18T22:27:10.100210+0000","last_clean":"2024-04-18T22:27:10.100210+0000","last_became_active":"2024-04-18T22:22:32.142649+0000","last_became_peered":"2024-04-18T22:22:32.142649+0000","last_unstale":"2024-04-18T22:27:10.100210+0000","last_undegraded":"2024-04-18T22:27:10.100210+0000","last_fullsized":"2024-04-18T22:27:10.100210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:10.099930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:10.099930+0000","last_clean_scrub_stamp":"2024-04-18T22:27:10.099930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:46:06.369939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:49.902373+0000","last_change":"2024-04-18T22:26:49.902426+0000","last_active":"2024-04-18T22:26:49.902373+0000","last_peered":"2024-04-18T22:26:49.902373+0000","last_clean":"2024-04-18T22:26:49.902373+0000","last_became_active":"2024-04-18T22:26:26.051801+0000","last_became_peered":"2024-04-18T22:26:26.051801+0000","last_unstale":"2024-04-18T22:26:49.902373+0000","last_undegraded":"2024-04-18T22:26:49.902373+0000","last_fullsized":"2024-04-18T22:26:49.902373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:49.902160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:49.902160+0000","last_clean_scrub_stamp":"2024-04-18T22:26:49.902160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:39:02.183017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:15.316418+0000","last_change":"2024-04-18T22:27:15.316473+0000","last_active":"2024-04-18T22:27:15.316418+0000","last_peered":"2024-04-18T22:27:15.316418+0000","last_clean":"2024-04-18T22:27:15.316418+0000","last_became_active":"2024-04-18T22:26:34.221378+0000","last_became_peered":"2024-04-18T22:26:34.221378+0000","last_unstale":"2024-04-18T22:27:15.316418+0000","last_undegraded":"2024-04-18T22:27:15.316418+0000","last_fullsized":"2024-04-18T22:27:15.316418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:15.316200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:15.316200+0000","last_clean_scrub_stamp":"2024-04-18T22:27:15.316200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:34:09.926108+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"83'12","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:55.680797+0000","last_change":"2024-04-18T22:26:55.680797+0000","last_active":"2024-04-18T22:26:55.680797+0000","last_peered":"2024-04-18T22:26:55.680797+0000","last_clean":"2024-04-18T22:26:55.680797+0000","last_became_active":"2024-04-18T22:26:37.248040+0000","last_became_peered":"2024-04-18T22:26:37.248040+0000","last_unstale":"2024-04-18T22:26:55.680797+0000","last_undegraded":"2024-04-18T22:26:55.680797+0000","last_fullsized":"2024-04-18T22:26:55.680797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'12","last_scrub_stamp":"2024-04-18T22:26:55.680541+0000","last_deep_scrub":"83'12","last_deep_scrub_stamp":"2024-04-18T22:26:55.680541+0000","last_clean_scrub_stamp":"2024-04-18T22:26:55.680541+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:20:05.389327+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"83'2","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:12.637143+0000","last_change":"2024-04-18T22:27:12.637203+0000","last_active":"2024-04-18T22:27:12.637143+0000","last_peered":"2024-04-18T22:27:12.637143+0000","last_clean":"2024-04-18T22:27:12.637143+0000","last_became_active":"2024-04-18T22:26:37.239997+0000","last_became_peered":"2024-04-18T22:26:37.239997+0000","last_unstale":"2024-04-18T22:27:12.637143+0000","last_undegraded":"2024-04-18T22:27:12.637143+0000","last_fullsized":"2024-04-18T22:27:12.637143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'2","last_scrub_stamp":"2024-04-18T22:27:12.636957+0000","last_deep_scrub":"83'2","last_deep_scrub_stamp":"2024-04-18T22:27:12.636957+0000","last_clean_scrub_stamp":"2024-04-18T22:27:12.636957+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:38:41.496971+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:12.742440+0000","last_change":"2024-04-18T22:27:12.742493+0000","last_active":"2024-04-18T22:27:12.742440+0000","last_peered":"2024-04-18T22:27:12.742440+0000","last_clean":"2024-04-18T22:27:12.742440+0000","last_became_active":"2024-04-18T22:26:26.056885+0000","last_became_peered":"2024-04-18T22:26:26.056885+0000","last_unstale":"2024-04-18T22:27:12.742440+0000","last_undegraded":"2024-04-18T22:27:12.742440+0000","last_fullsized":"2024-04-18T22:27:12.742440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:12.742184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:12.742184+0000","last_clean_scrub_stamp":"2024-04-18T22:27:12.742184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:08:35.726911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"83'3","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:56.760117+0000","last_change":"2024-04-18T22:26:56.760178+0000","last_active":"2024-04-18T22:26:56.760117+0000","last_peered":"2024-04-18T22:26:56.760117+0000","last_clean":"2024-04-18T22:26:56.760117+0000","last_became_active":"2024-04-18T22:26:26.055871+0000","last_became_peered":"2024-04-18T22:26:26.055871+0000","last_unstale":"2024-04-18T22:26:56.760117+0000","last_undegraded":"2024-04-18T22:26:56.760117+0000","last_fullsized":"2024-04-18T22:26:56.760117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:56.759936+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:56.759936+0000","last_clean_scrub_stamp":"2024-04-18T22:26:56.759936+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:17:43.854967+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:06.751283+0000","last_change":"2024-04-18T22:27:06.751413+0000","last_active":"2024-04-18T22:27:06.751283+0000","last_peered":"2024-04-18T22:27:06.751283+0000","last_clean":"2024-04-18T22:27:06.751283+0000","last_became_active":"2024-04-18T22:26:26.034143+0000","last_became_peered":"2024-04-18T22:26:26.034143+0000","last_unstale":"2024-04-18T22:27:06.751283+0000","last_undegraded":"2024-04-18T22:27:06.751283+0000","last_fullsized":"2024-04-18T22:27:06.751283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:06.751053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:06.751053+0000","last_clean_scrub_stamp":"2024-04-18T22:27:06.751053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:04:55.998477+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"83'2","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:22.789532+0000","last_change":"2024-04-18T22:27:22.789595+0000","last_active":"2024-04-18T22:27:22.789532+0000","last_peered":"2024-04-18T22:27:22.789532+0000","last_clean":"2024-04-18T22:27:22.789532+0000","last_became_active":"2024-04-18T22:26:37.254494+0000","last_became_peered":"2024-04-18T22:26:37.254494+0000","last_unstale":"2024-04-18T22:27:22.789532+0000","last_undegraded":"2024-04-18T22:27:22.789532+0000","last_fullsized":"2024-04-18T22:27:22.789532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'2","last_scrub_stamp":"2024-04-18T22:27:22.789356+0000","last_deep_scrub":"83'2","last_deep_scrub_stamp":"2024-04-18T22:27:22.789356+0000","last_clean_scrub_stamp":"2024-04-18T22:27:22.789356+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:25:48.687320+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:07.761420+0000","last_change":"2024-04-18T22:27:07.761479+0000","last_active":"2024-04-18T22:27:07.761420+0000","last_peered":"2024-04-18T22:27:07.761420+0000","last_clean":"2024-04-18T22:27:07.761420+0000","last_became_active":"2024-04-18T22:26:37.248925+0000","last_became_peered":"2024-04-18T22:26:37.248925+0000","last_unstale":"2024-04-18T22:27:07.761420+0000","last_undegraded":"2024-04-18T22:27:07.761420+0000","last_fullsized":"2024-04-18T22:27:07.761420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:07.761233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:07.761233+0000","last_clean_scrub_stamp":"2024-04-18T22:27:07.761233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:34:02.371141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:56.724345+0000","last_change":"2024-04-18T22:26:56.724398+0000","last_active":"2024-04-18T22:26:56.724345+0000","last_peered":"2024-04-18T22:26:56.724345+0000","last_clean":"2024-04-18T22:26:56.724345+0000","last_became_active":"2024-04-18T22:26:37.245464+0000","last_became_peered":"2024-04-18T22:26:37.245464+0000","last_unstale":"2024-04-18T22:26:56.724345+0000","last_undegraded":"2024-04-18T22:26:56.724345+0000","last_fullsized":"2024-04-18T22:26:56.724345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:56.724139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:56.724139+0000","last_clean_scrub_stamp":"2024-04-18T22:26:56.724139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:23:42.605818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:08.102176+0000","last_change":"2024-04-18T22:27:08.102222+0000","last_active":"2024-04-18T22:27:08.102176+0000","last_peered":"2024-04-18T22:27:08.102176+0000","last_clean":"2024-04-18T22:27:08.102176+0000","last_became_active":"2024-04-18T22:22:32.142833+0000","last_became_peered":"2024-04-18T22:22:32.142833+0000","last_unstale":"2024-04-18T22:27:08.102176+0000","last_undegraded":"2024-04-18T22:27:08.102176+0000","last_fullsized":"2024-04-18T22:27:08.102176+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:08.102006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:08.102006+0000","last_clean_scrub_stamp":"2024-04-18T22:27:08.102006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:26:00.007900+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"83'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:17.258608+0000","last_change":"2024-04-18T22:27:17.258673+0000","last_active":"2024-04-18T22:27:17.258608+0000","last_peered":"2024-04-18T22:27:17.258608+0000","last_clean":"2024-04-18T22:27:17.258608+0000","last_became_active":"2024-04-18T22:26:34.217122+0000","last_became_peered":"2024-04-18T22:26:34.217122+0000","last_unstale":"2024-04-18T22:27:17.258608+0000","last_undegraded":"2024-04-18T22:27:17.258608+0000","last_fullsized":"2024-04-18T22:27:17.258608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:27:17.258426+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:27:17.258426+0000","last_clean_scrub_stamp":"2024-04-18T22:27:17.258426+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:03.458497+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:52.713924+0000","last_change":"2024-04-18T22:26:52.713924+0000","last_active":"2024-04-18T22:26:52.713924+0000","last_peered":"2024-04-18T22:26:52.713924+0000","last_clean":"2024-04-18T22:26:52.713924+0000","last_became_active":"2024-04-18T22:26:37.251174+0000","last_became_peered":"2024-04-18T22:26:37.251174+0000","last_unstale":"2024-04-18T22:26:52.713924+0000","last_undegraded":"2024-04-18T22:26:52.713924+0000","last_fullsized":"2024-04-18T22:26:52.713924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:52.713672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:52.713672+0000","last_clean_scrub_stamp":"2024-04-18T22:26:52.713672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T05:18:16.911539+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.757847+0000","last_change":"2024-04-18T22:27:04.757899+0000","last_active":"2024-04-18T22:27:04.757847+0000","last_peered":"2024-04-18T22:27:04.757847+0000","last_clean":"2024-04-18T22:27:04.757847+0000","last_became_active":"2024-04-18T22:26:37.236919+0000","last_became_peered":"2024-04-18T22:26:37.236919+0000","last_unstale":"2024-04-18T22:27:04.757847+0000","last_undegraded":"2024-04-18T22:27:04.757847+0000","last_fullsized":"2024-04-18T22:27:04.757847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.757657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.757657+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.757657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:09:57.845790+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:23.776985+0000","last_change":"2024-04-18T22:27:23.777037+0000","last_active":"2024-04-18T22:27:23.776985+0000","last_peered":"2024-04-18T22:27:23.776985+0000","last_clean":"2024-04-18T22:27:23.776985+0000","last_became_active":"2024-04-18T22:26:37.254665+0000","last_became_peered":"2024-04-18T22:26:37.254665+0000","last_unstale":"2024-04-18T22:27:23.776985+0000","last_undegraded":"2024-04-18T22:27:23.776985+0000","last_fullsized":"2024-04-18T22:27:23.776985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:23.776801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:23.776801+0000","last_clean_scrub_stamp":"2024-04-18T22:27:23.776801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:02:36.102379+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:48.881120+0000","last_change":"2024-04-18T22:26:48.881171+0000","last_active":"2024-04-18T22:26:48.881120+0000","last_peered":"2024-04-18T22:26:48.881120+0000","last_clean":"2024-04-18T22:26:48.881120+0000","last_became_active":"2024-04-18T22:26:26.052235+0000","last_became_peered":"2024-04-18T22:26:26.052235+0000","last_unstale":"2024-04-18T22:26:48.881120+0000","last_undegraded":"2024-04-18T22:26:48.881120+0000","last_fullsized":"2024-04-18T22:26:48.881120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:48.880922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:48.880922+0000","last_clean_scrub_stamp":"2024-04-18T22:26:48.880922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:03:22.286281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:16.681741+0000","last_change":"2024-04-18T22:27:16.681792+0000","last_active":"2024-04-18T22:27:16.681741+0000","last_peered":"2024-04-18T22:27:16.681741+0000","last_clean":"2024-04-18T22:27:16.681741+0000","last_became_active":"2024-04-18T22:26:37.242106+0000","last_became_peered":"2024-04-18T22:26:37.242106+0000","last_unstale":"2024-04-18T22:27:16.681741+0000","last_undegraded":"2024-04-18T22:27:16.681741+0000","last_fullsized":"2024-04-18T22:27:16.681741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:16.681557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:16.681557+0000","last_clean_scrub_stamp":"2024-04-18T22:27:16.681557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:44:42.781184+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:11.755960+0000","last_change":"2024-04-18T22:27:11.756011+0000","last_active":"2024-04-18T22:27:11.755960+0000","last_peered":"2024-04-18T22:27:11.755960+0000","last_clean":"2024-04-18T22:27:11.755960+0000","last_became_active":"2024-04-18T22:26:26.035383+0000","last_became_peered":"2024-04-18T22:26:26.035383+0000","last_unstale":"2024-04-18T22:27:11.755960+0000","last_undegraded":"2024-04-18T22:27:11.755960+0000","last_fullsized":"2024-04-18T22:27:11.755960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:11.755762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:11.755762+0000","last_clean_scrub_stamp":"2024-04-18T22:27:11.755762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:05:08.211874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:48.191395+0000","last_change":"2024-04-18T22:26:48.191395+0000","last_active":"2024-04-18T22:26:48.191395+0000","last_peered":"2024-04-18T22:26:48.191395+0000","last_clean":"2024-04-18T22:26:48.191395+0000","last_became_active":"2024-04-18T22:26:26.068131+0000","last_became_peered":"2024-04-18T22:26:26.068131+0000","last_unstale":"2024-04-18T22:26:48.191395+0000","last_undegraded":"2024-04-18T22:26:48.191395+0000","last_fullsized":"2024-04-18T22:26:48.191395+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:48.191250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:48.191250+0000","last_clean_scrub_stamp":"2024-04-18T22:26:48.191250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:32:54.665666+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"83'159","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.848528+0000","last_change":"2024-04-18T22:27:00.848605+0000","last_active":"2024-04-18T22:27:00.848528+0000","last_peered":"2024-04-18T22:27:00.848528+0000","last_clean":"2024-04-18T22:27:00.848528+0000","last_became_active":"2024-04-18T22:26:37.242110+0000","last_became_peered":"2024-04-18T22:26:37.242110+0000","last_unstale":"2024-04-18T22:27:00.848528+0000","last_undegraded":"2024-04-18T22:27:00.848528+0000","last_fullsized":"2024-04-18T22:27:00.848528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'159","last_scrub_stamp":"2024-04-18T22:27:00.848340+0000","last_deep_scrub":"83'159","last_deep_scrub_stamp":"2024-04-18T22:27:00.848340+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.848340+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:25:52.754235+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:01.772936+0000","last_change":"2024-04-18T22:27:01.772987+0000","last_active":"2024-04-18T22:27:01.772936+0000","last_peered":"2024-04-18T22:27:01.772936+0000","last_clean":"2024-04-18T22:27:01.772936+0000","last_became_active":"2024-04-18T22:26:37.241190+0000","last_became_peered":"2024-04-18T22:26:37.241190+0000","last_unstale":"2024-04-18T22:27:01.772936+0000","last_undegraded":"2024-04-18T22:27:01.772936+0000","last_fullsized":"2024-04-18T22:27:01.772936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:01.772738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:01.772738+0000","last_clean_scrub_stamp":"2024-04-18T22:27:01.772738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:05:02.982896+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.044809+0000","last_change":"2024-04-18T22:26:58.044864+0000","last_active":"2024-04-18T22:26:58.044809+0000","last_peered":"2024-04-18T22:26:58.044809+0000","last_clean":"2024-04-18T22:26:58.044809+0000","last_became_active":"2024-04-18T22:26:34.215318+0000","last_became_peered":"2024-04-18T22:26:34.215318+0000","last_unstale":"2024-04-18T22:26:58.044809+0000","last_undegraded":"2024-04-18T22:26:58.044809+0000","last_fullsized":"2024-04-18T22:26:58.044809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.044612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.044612+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.044612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:48:11.405080+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:48.745353+0000","last_change":"2024-04-18T22:26:48.745407+0000","last_active":"2024-04-18T22:26:48.745353+0000","last_peered":"2024-04-18T22:26:48.745353+0000","last_clean":"2024-04-18T22:26:48.745353+0000","last_became_active":"2024-04-18T22:26:37.247443+0000","last_became_peered":"2024-04-18T22:26:37.247443+0000","last_unstale":"2024-04-18T22:26:48.745353+0000","last_undegraded":"2024-04-18T22:26:48.745353+0000","last_fullsized":"2024-04-18T22:26:48.745353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:48.745171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:48.745171+0000","last_clean_scrub_stamp":"2024-04-18T22:26:48.745171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:05:58.049454+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:15.654721+0000","last_change":"2024-04-18T22:27:15.654777+0000","last_active":"2024-04-18T22:27:15.654721+0000","last_peered":"2024-04-18T22:27:15.654721+0000","last_clean":"2024-04-18T22:27:15.654721+0000","last_became_active":"2024-04-18T22:26:37.237065+0000","last_became_peered":"2024-04-18T22:26:37.237065+0000","last_unstale":"2024-04-18T22:27:15.654721+0000","last_undegraded":"2024-04-18T22:27:15.654721+0000","last_fullsized":"2024-04-18T22:27:15.654721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:15.654517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:15.654517+0000","last_clean_scrub_stamp":"2024-04-18T22:27:15.654517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:25:45.484155+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:04.007241+0000","last_change":"2024-04-18T22:27:04.007293+0000","last_active":"2024-04-18T22:27:04.007241+0000","last_peered":"2024-04-18T22:27:04.007241+0000","last_clean":"2024-04-18T22:27:04.007241+0000","last_became_active":"2024-04-18T22:22:31.137535+0000","last_became_peered":"2024-04-18T22:22:31.137535+0000","last_unstale":"2024-04-18T22:27:04.007241+0000","last_undegraded":"2024-04-18T22:27:04.007241+0000","last_fullsized":"2024-04-18T22:27:04.007241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:04.007052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:04.007052+0000","last_clean_scrub_stamp":"2024-04-18T22:27:04.007052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:28:44.608219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:16.774990+0000","last_change":"2024-04-18T22:27:16.775041+0000","last_active":"2024-04-18T22:27:16.774990+0000","last_peered":"2024-04-18T22:27:16.774990+0000","last_clean":"2024-04-18T22:27:16.774990+0000","last_became_active":"2024-04-18T22:26:34.195495+0000","last_became_peered":"2024-04-18T22:26:34.195495+0000","last_unstale":"2024-04-18T22:27:16.774990+0000","last_undegraded":"2024-04-18T22:27:16.774990+0000","last_fullsized":"2024-04-18T22:27:16.774990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:16.774790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:16.774790+0000","last_clean_scrub_stamp":"2024-04-18T22:27:16.774790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:07.227021+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:57.052337+0000","last_change":"2024-04-18T22:26:57.052395+0000","last_active":"2024-04-18T22:26:57.052337+0000","last_peered":"2024-04-18T22:26:57.052337+0000","last_clean":"2024-04-18T22:26:57.052337+0000","last_became_active":"2024-04-18T22:26:26.066910+0000","last_became_peered":"2024-04-18T22:26:26.066910+0000","last_unstale":"2024-04-18T22:26:57.052337+0000","last_undegraded":"2024-04-18T22:26:57.052337+0000","last_fullsized":"2024-04-18T22:26:57.052337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:57.052137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:57.052137+0000","last_clean_scrub_stamp":"2024-04-18T22:26:57.052137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:06.647950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:46.866157+0000","last_change":"2024-04-18T22:26:46.866228+0000","last_active":"2024-04-18T22:26:46.866157+0000","last_peered":"2024-04-18T22:26:46.866157+0000","last_clean":"2024-04-18T22:26:46.866157+0000","last_became_active":"2024-04-18T22:26:26.057876+0000","last_became_peered":"2024-04-18T22:26:26.057876+0000","last_unstale":"2024-04-18T22:26:46.866157+0000","last_undegraded":"2024-04-18T22:26:46.866157+0000","last_fullsized":"2024-04-18T22:26:46.866157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:46.865964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:46.865964+0000","last_clean_scrub_stamp":"2024-04-18T22:26:46.865964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:50:11.432855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:16.287738+0000","last_change":"2024-04-18T22:27:16.287783+0000","last_active":"2024-04-18T22:27:16.287738+0000","last_peered":"2024-04-18T22:27:16.287738+0000","last_clean":"2024-04-18T22:27:16.287738+0000","last_became_active":"2024-04-18T22:26:34.222769+0000","last_became_peered":"2024-04-18T22:26:34.222769+0000","last_unstale":"2024-04-18T22:27:16.287738+0000","last_undegraded":"2024-04-18T22:27:16.287738+0000","last_fullsized":"2024-04-18T22:27:16.287738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:16.287587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:16.287587+0000","last_clean_scrub_stamp":"2024-04-18T22:27:16.287587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:07:08.778103+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:11.668654+0000","last_change":"2024-04-18T22:27:11.668706+0000","last_active":"2024-04-18T22:27:11.668654+0000","last_peered":"2024-04-18T22:27:11.668654+0000","last_clean":"2024-04-18T22:27:11.668654+0000","last_became_active":"2024-04-18T22:26:37.235938+0000","last_became_peered":"2024-04-18T22:26:37.235938+0000","last_unstale":"2024-04-18T22:27:11.668654+0000","last_undegraded":"2024-04-18T22:27:11.668654+0000","last_fullsized":"2024-04-18T22:27:11.668654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:11.668472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:11.668472+0000","last_clean_scrub_stamp":"2024-04-18T22:27:11.668472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:03:21.661678+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:58.752829+0000","last_change":"2024-04-18T22:26:58.752885+0000","last_active":"2024-04-18T22:26:58.752829+0000","last_peered":"2024-04-18T22:26:58.752829+0000","last_clean":"2024-04-18T22:26:58.752829+0000","last_became_active":"2024-04-18T22:26:37.241882+0000","last_became_peered":"2024-04-18T22:26:37.241882+0000","last_unstale":"2024-04-18T22:26:58.752829+0000","last_undegraded":"2024-04-18T22:26:58.752829+0000","last_fullsized":"2024-04-18T22:26:58.752829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:58.752607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:58.752607+0000","last_clean_scrub_stamp":"2024-04-18T22:26:58.752607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:35:57.342452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:12.148174+0000","last_change":"2024-04-18T22:27:12.148233+0000","last_active":"2024-04-18T22:27:12.148174+0000","last_peered":"2024-04-18T22:27:12.148174+0000","last_clean":"2024-04-18T22:27:12.148174+0000","last_became_active":"2024-04-18T22:26:34.219746+0000","last_became_peered":"2024-04-18T22:26:34.219746+0000","last_unstale":"2024-04-18T22:27:12.148174+0000","last_undegraded":"2024-04-18T22:27:12.148174+0000","last_fullsized":"2024-04-18T22:27:12.148174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:12.147958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:12.147958+0000","last_clean_scrub_stamp":"2024-04-18T22:27:12.147958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:34:34.463858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:24.816619+0000","last_change":"2024-04-18T22:27:24.816673+0000","last_active":"2024-04-18T22:27:24.816619+0000","last_peered":"2024-04-18T22:27:24.816619+0000","last_clean":"2024-04-18T22:27:24.816619+0000","last_became_active":"2024-04-18T22:26:37.256374+0000","last_became_peered":"2024-04-18T22:26:37.256374+0000","last_unstale":"2024-04-18T22:27:24.816619+0000","last_undegraded":"2024-04-18T22:27:24.816619+0000","last_fullsized":"2024-04-18T22:27:24.816619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:24.816405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:24.816405+0000","last_clean_scrub_stamp":"2024-04-18T22:27:24.816405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:21:40.528723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:55.063289+0000","last_change":"2024-04-18T22:26:55.063345+0000","last_active":"2024-04-18T22:26:55.063289+0000","last_peered":"2024-04-18T22:26:55.063289+0000","last_clean":"2024-04-18T22:26:55.063289+0000","last_became_active":"2024-04-18T22:26:26.067984+0000","last_became_peered":"2024-04-18T22:26:26.067984+0000","last_unstale":"2024-04-18T22:26:55.063289+0000","last_undegraded":"2024-04-18T22:26:55.063289+0000","last_fullsized":"2024-04-18T22:26:55.063289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:55.063087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:55.063087+0000","last_clean_scrub_stamp":"2024-04-18T22:26:55.063087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:12:15.782434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:51.869133+0000","last_change":"2024-04-18T22:26:51.869187+0000","last_active":"2024-04-18T22:26:51.869133+0000","last_peered":"2024-04-18T22:26:51.869133+0000","last_clean":"2024-04-18T22:26:51.869133+0000","last_became_active":"2024-04-18T22:26:26.049332+0000","last_became_peered":"2024-04-18T22:26:26.049332+0000","last_unstale":"2024-04-18T22:26:51.869133+0000","last_undegraded":"2024-04-18T22:26:51.869133+0000","last_fullsized":"2024-04-18T22:26:51.869133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:51.868951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:51.868951+0000","last_clean_scrub_stamp":"2024-04-18T22:26:51.868951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:16:34.019162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:52.119518+0000","last_change":"2024-04-18T22:26:52.119573+0000","last_active":"2024-04-18T22:26:52.119518+0000","last_peered":"2024-04-18T22:26:52.119518+0000","last_clean":"2024-04-18T22:26:52.119518+0000","last_became_active":"2024-04-18T22:26:34.213842+0000","last_became_peered":"2024-04-18T22:26:34.213842+0000","last_unstale":"2024-04-18T22:26:52.119518+0000","last_undegraded":"2024-04-18T22:26:52.119518+0000","last_fullsized":"2024-04-18T22:26:52.119518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:52.119319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:52.119319+0000","last_clean_scrub_stamp":"2024-04-18T22:26:52.119319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:19:48.419694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:14.716983+0000","last_change":"2024-04-18T22:27:14.717035+0000","last_active":"2024-04-18T22:27:14.716983+0000","last_peered":"2024-04-18T22:27:14.716983+0000","last_clean":"2024-04-18T22:27:14.716983+0000","last_became_active":"2024-04-18T22:26:34.186206+0000","last_became_peered":"2024-04-18T22:26:34.186206+0000","last_unstale":"2024-04-18T22:27:14.716983+0000","last_undegraded":"2024-04-18T22:27:14.716983+0000","last_fullsized":"2024-04-18T22:27:14.716983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:14.716778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:14.716778+0000","last_clean_scrub_stamp":"2024-04-18T22:27:14.716778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:28:34.677746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:55.801445+0000","last_change":"2024-04-18T22:26:55.801497+0000","last_active":"2024-04-18T22:26:55.801445+0000","last_peered":"2024-04-18T22:26:55.801445+0000","last_clean":"2024-04-18T22:26:55.801445+0000","last_became_active":"2024-04-18T22:26:26.049519+0000","last_became_peered":"2024-04-18T22:26:26.049519+0000","last_unstale":"2024-04-18T22:26:55.801445+0000","last_undegraded":"2024-04-18T22:26:55.801445+0000","last_fullsized":"2024-04-18T22:26:55.801445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:55.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:55.801255+0000","last_clean_scrub_stamp":"2024-04-18T22:26:55.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:40:07.910755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"83'3","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:50.118728+0000","last_change":"2024-04-18T22:26:50.118728+0000","last_active":"2024-04-18T22:26:50.118728+0000","last_peered":"2024-04-18T22:26:50.118728+0000","last_clean":"2024-04-18T22:26:50.118728+0000","last_became_active":"2024-04-18T22:26:26.069775+0000","last_became_peered":"2024-04-18T22:26:26.069775+0000","last_unstale":"2024-04-18T22:26:50.118728+0000","last_undegraded":"2024-04-18T22:26:50.118728+0000","last_fullsized":"2024-04-18T22:26:50.118728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:50.118565+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:50.118565+0000","last_clean_scrub_stamp":"2024-04-18T22:26:50.118565+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:08:52.966118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:56.030184+0000","last_change":"2024-04-18T22:26:56.030238+0000","last_active":"2024-04-18T22:26:56.030184+0000","last_peered":"2024-04-18T22:26:56.030184+0000","last_clean":"2024-04-18T22:26:56.030184+0000","last_became_active":"2024-04-18T22:26:34.215100+0000","last_became_peered":"2024-04-18T22:26:34.215100+0000","last_unstale":"2024-04-18T22:26:56.030184+0000","last_undegraded":"2024-04-18T22:26:56.030184+0000","last_fullsized":"2024-04-18T22:26:56.030184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:56.029987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:56.029987+0000","last_clean_scrub_stamp":"2024-04-18T22:26:56.029987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:52:23.546855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:13.189181+0000","last_change":"2024-04-18T22:27:13.189238+0000","last_active":"2024-04-18T22:27:13.189181+0000","last_peered":"2024-04-18T22:27:13.189181+0000","last_clean":"2024-04-18T22:27:13.189181+0000","last_became_active":"2024-04-18T22:26:34.220572+0000","last_became_peered":"2024-04-18T22:26:34.220572+0000","last_unstale":"2024-04-18T22:27:13.189181+0000","last_undegraded":"2024-04-18T22:27:13.189181+0000","last_fullsized":"2024-04-18T22:27:13.189181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:13.188988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:13.188988+0000","last_clean_scrub_stamp":"2024-04-18T22:27:13.188988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:10:07.965545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"83'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:18.238359+0000","last_change":"2024-04-18T22:27:18.238420+0000","last_active":"2024-04-18T22:27:18.238359+0000","last_peered":"2024-04-18T22:27:18.238359+0000","last_clean":"2024-04-18T22:27:18.238359+0000","last_became_active":"2024-04-18T22:26:34.221981+0000","last_became_peered":"2024-04-18T22:26:34.221981+0000","last_unstale":"2024-04-18T22:27:18.238359+0000","last_undegraded":"2024-04-18T22:27:18.238359+0000","last_fullsized":"2024-04-18T22:27:18.238359+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'2","last_scrub_stamp":"2024-04-18T22:27:18.238169+0000","last_deep_scrub":"83'2","last_deep_scrub_stamp":"2024-04-18T22:27:18.238169+0000","last_clean_scrub_stamp":"2024-04-18T22:27:18.238169+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:41:17.695604+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:13.615201+0000","last_change":"2024-04-18T22:27:13.615255+0000","last_active":"2024-04-18T22:27:13.615201+0000","last_peered":"2024-04-18T22:27:13.615201+0000","last_clean":"2024-04-18T22:27:13.615201+0000","last_became_active":"2024-04-18T22:26:37.240126+0000","last_became_peered":"2024-04-18T22:26:37.240126+0000","last_unstale":"2024-04-18T22:27:13.615201+0000","last_undegraded":"2024-04-18T22:27:13.615201+0000","last_fullsized":"2024-04-18T22:27:13.615201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:13.614993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:13.614993+0000","last_clean_scrub_stamp":"2024-04-18T22:27:13.614993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T07:19:28.381784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:01.702948+0000","last_change":"2024-04-18T22:27:01.703002+0000","last_active":"2024-04-18T22:27:01.702948+0000","last_peered":"2024-04-18T22:27:01.702948+0000","last_clean":"2024-04-18T22:27:01.702948+0000","last_became_active":"2024-04-18T22:26:26.049057+0000","last_became_peered":"2024-04-18T22:26:26.049057+0000","last_unstale":"2024-04-18T22:27:01.702948+0000","last_undegraded":"2024-04-18T22:27:01.702948+0000","last_fullsized":"2024-04-18T22:27:01.702948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:01.702757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:01.702757+0000","last_clean_scrub_stamp":"2024-04-18T22:27:01.702757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:31:51.781627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:49.724754+0000","last_change":"2024-04-18T22:26:49.724754+0000","last_active":"2024-04-18T22:26:49.724754+0000","last_peered":"2024-04-18T22:26:49.724754+0000","last_clean":"2024-04-18T22:26:49.724754+0000","last_became_active":"2024-04-18T22:26:37.250474+0000","last_became_peered":"2024-04-18T22:26:37.250474+0000","last_unstale":"2024-04-18T22:26:49.724754+0000","last_undegraded":"2024-04-18T22:26:49.724754+0000","last_fullsized":"2024-04-18T22:26:49.724754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:49.724511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:49.724511+0000","last_clean_scrub_stamp":"2024-04-18T22:26:49.724511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:28:29.140137+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"83'10","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:06.752692+0000","last_change":"2024-04-18T22:27:06.752692+0000","last_active":"2024-04-18T22:27:06.752692+0000","last_peered":"2024-04-18T22:27:06.752692+0000","last_clean":"2024-04-18T22:27:06.752692+0000","last_became_active":"2024-04-18T22:26:37.244435+0000","last_became_peered":"2024-04-18T22:26:37.244435+0000","last_unstale":"2024-04-18T22:27:06.752692+0000","last_undegraded":"2024-04-18T22:27:06.752692+0000","last_fullsized":"2024-04-18T22:27:06.752692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-04-18T22:27:06.752437+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-04-18T22:27:06.752437+0000","last_clean_scrub_stamp":"2024-04-18T22:27:06.752437+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T03:23:30.392357+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:00.990708+0000","last_change":"2024-04-18T22:27:00.990708+0000","last_active":"2024-04-18T22:27:00.990708+0000","last_peered":"2024-04-18T22:27:00.990708+0000","last_clean":"2024-04-18T22:27:00.990708+0000","last_became_active":"2024-04-18T22:26:26.058530+0000","last_became_peered":"2024-04-18T22:26:26.058530+0000","last_unstale":"2024-04-18T22:27:00.990708+0000","last_undegraded":"2024-04-18T22:27:00.990708+0000","last_fullsized":"2024-04-18T22:27:00.990708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:00.990461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:00.990461+0000","last_clean_scrub_stamp":"2024-04-18T22:27:00.990461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:40.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"83'5","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:47.863158+0000","last_change":"2024-04-18T22:26:47.863223+0000","last_active":"2024-04-18T22:26:47.863158+0000","last_peered":"2024-04-18T22:26:47.863158+0000","last_clean":"2024-04-18T22:26:47.863158+0000","last_became_active":"2024-04-18T22:26:26.057531+0000","last_became_peered":"2024-04-18T22:26:26.057531+0000","last_unstale":"2024-04-18T22:26:47.863158+0000","last_undegraded":"2024-04-18T22:26:47.863158+0000","last_fullsized":"2024-04-18T22:26:47.863158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-04-18T22:26:47.862971+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-04-18T22:26:47.862971+0000","last_clean_scrub_stamp":"2024-04-18T22:26:47.862971+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:01.341015+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:02.787041+0000","last_change":"2024-04-18T22:27:02.787041+0000","last_active":"2024-04-18T22:27:02.787041+0000","last_peered":"2024-04-18T22:27:02.787041+0000","last_clean":"2024-04-18T22:27:02.787041+0000","last_became_active":"2024-04-18T22:26:37.239536+0000","last_became_peered":"2024-04-18T22:26:37.239536+0000","last_unstale":"2024-04-18T22:27:02.787041+0000","last_undegraded":"2024-04-18T22:27:02.787041+0000","last_fullsized":"2024-04-18T22:27:02.787041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:02.786795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:02.786795+0000","last_clean_scrub_stamp":"2024-04-18T22:27:02.786795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:45:59.056804+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:07.029507+0000","last_change":"2024-04-18T22:27:07.029507+0000","last_active":"2024-04-18T22:27:07.029507+0000","last_peered":"2024-04-18T22:27:07.029507+0000","last_clean":"2024-04-18T22:27:07.029507+0000","last_became_active":"2024-04-18T22:22:32.142512+0000","last_became_peered":"2024-04-18T22:22:32.142512+0000","last_unstale":"2024-04-18T22:27:07.029507+0000","last_undegraded":"2024-04-18T22:27:07.029507+0000","last_fullsized":"2024-04-18T22:27:07.029507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:07.029213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:07.029213+0000","last_clean_scrub_stamp":"2024-04-18T22:27:07.029213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T00:46:53.656490+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:05.997942+0000","last_change":"2024-04-18T22:27:05.997942+0000","last_active":"2024-04-18T22:27:05.997942+0000","last_peered":"2024-04-18T22:27:05.997942+0000","last_clean":"2024-04-18T22:27:05.997942+0000","last_became_active":"2024-04-18T22:22:31.140423+0000","last_became_peered":"2024-04-18T22:22:31.140423+0000","last_unstale":"2024-04-18T22:27:05.997942+0000","last_undegraded":"2024-04-18T22:27:05.997942+0000","last_fullsized":"2024-04-18T22:27:05.997942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:05.997695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:05.997695+0000","last_clean_scrub_stamp":"2024-04-18T22:27:05.997695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:36:04.587540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:47.051716+0000","last_change":"2024-04-18T22:26:47.051774+0000","last_active":"2024-04-18T22:26:47.051716+0000","last_peered":"2024-04-18T22:26:47.051716+0000","last_clean":"2024-04-18T22:26:47.051716+0000","last_became_active":"2024-04-18T22:26:34.218910+0000","last_became_peered":"2024-04-18T22:26:34.218910+0000","last_unstale":"2024-04-18T22:26:47.051716+0000","last_undegraded":"2024-04-18T22:26:47.051716+0000","last_fullsized":"2024-04-18T22:26:47.051716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:47.051489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:47.051489+0000","last_clean_scrub_stamp":"2024-04-18T22:26:47.051489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:53:21.437741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:50.993560+0000","last_change":"2024-04-18T22:26:50.993683+0000","last_active":"2024-04-18T22:26:50.993560+0000","last_peered":"2024-04-18T22:26:50.993560+0000","last_clean":"2024-04-18T22:26:50.993560+0000","last_became_active":"2024-04-18T22:26:26.052448+0000","last_became_peered":"2024-04-18T22:26:26.052448+0000","last_unstale":"2024-04-18T22:26:50.993560+0000","last_undegraded":"2024-04-18T22:26:50.993560+0000","last_fullsized":"2024-04-18T22:26:50.993560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:50.993315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:50.993315+0000","last_clean_scrub_stamp":"2024-04-18T22:26:50.993315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:11:24.120390+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:54.071697+0000","last_change":"2024-04-18T22:26:54.071750+0000","last_active":"2024-04-18T22:26:54.071697+0000","last_peered":"2024-04-18T22:26:54.071697+0000","last_clean":"2024-04-18T22:26:54.071697+0000","last_became_active":"2024-04-18T22:26:34.214835+0000","last_became_peered":"2024-04-18T22:26:34.214835+0000","last_unstale":"2024-04-18T22:26:54.071697+0000","last_undegraded":"2024-04-18T22:26:54.071697+0000","last_fullsized":"2024-04-18T22:26:54.071697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:54.071502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:54.071502+0000","last_clean_scrub_stamp":"2024-04-18T22:26:54.071502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:57:11.686559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:03.050790+0000","last_change":"2024-04-18T22:27:03.050845+0000","last_active":"2024-04-18T22:27:03.050790+0000","last_peered":"2024-04-18T22:27:03.050790+0000","last_clean":"2024-04-18T22:27:03.050790+0000","last_became_active":"2024-04-18T22:22:32.143671+0000","last_became_peered":"2024-04-18T22:22:32.143671+0000","last_unstale":"2024-04-18T22:27:03.050790+0000","last_undegraded":"2024-04-18T22:27:03.050790+0000","last_fullsized":"2024-04-18T22:27:03.050790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:03.050604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:03.050604+0000","last_clean_scrub_stamp":"2024-04-18T22:27:03.050604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:20:12.759390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:02.003535+0000","last_change":"2024-04-18T22:27:02.003589+0000","last_active":"2024-04-18T22:27:02.003535+0000","last_peered":"2024-04-18T22:27:02.003535+0000","last_clean":"2024-04-18T22:27:02.003535+0000","last_became_active":"2024-04-18T22:26:34.216785+0000","last_became_peered":"2024-04-18T22:26:34.216785+0000","last_unstale":"2024-04-18T22:27:02.003535+0000","last_undegraded":"2024-04-18T22:27:02.003535+0000","last_fullsized":"2024-04-18T22:27:02.003535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:02.003343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:02.003343+0000","last_clean_scrub_stamp":"2024-04-18T22:27:02.003343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T08:42:06.603570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:02.723432+0000","last_change":"2024-04-18T22:27:02.723485+0000","last_active":"2024-04-18T22:27:02.723432+0000","last_peered":"2024-04-18T22:27:02.723432+0000","last_clean":"2024-04-18T22:27:02.723432+0000","last_became_active":"2024-04-18T22:26:26.056679+0000","last_became_peered":"2024-04-18T22:26:26.056679+0000","last_unstale":"2024-04-18T22:27:02.723432+0000","last_undegraded":"2024-04-18T22:27:02.723432+0000","last_fullsized":"2024-04-18T22:27:02.723432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:02.723244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:02.723244+0000","last_clean_scrub_stamp":"2024-04-18T22:27:02.723244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:31:34.496835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"17'1","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:08.711167+0000","last_change":"2024-04-18T22:27:08.711231+0000","last_active":"2024-04-18T22:27:08.711167+0000","last_peered":"2024-04-18T22:27:08.711167+0000","last_clean":"2024-04-18T22:27:08.711167+0000","last_became_active":"2024-04-18T22:26:26.035090+0000","last_became_peered":"2024-04-18T22:26:26.035090+0000","last_unstale":"2024-04-18T22:27:08.711167+0000","last_undegraded":"2024-04-18T22:27:08.711167+0000","last_fullsized":"2024-04-18T22:27:08.711167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'1","last_scrub_stamp":"2024-04-18T22:27:08.710968+0000","last_deep_scrub":"17'1","last_deep_scrub_stamp":"2024-04-18T22:27:08.710968+0000","last_clean_scrub_stamp":"2024-04-18T22:27:08.710968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:14:12.919350+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:18.725232+0000","last_change":"2024-04-18T22:27:18.725289+0000","last_active":"2024-04-18T22:27:18.725232+0000","last_peered":"2024-04-18T22:27:18.725232+0000","last_clean":"2024-04-18T22:27:18.725232+0000","last_became_active":"2024-04-18T22:26:34.195250+0000","last_became_peered":"2024-04-18T22:26:34.195250+0000","last_unstale":"2024-04-18T22:27:18.725232+0000","last_undegraded":"2024-04-18T22:27:18.725232+0000","last_fullsized":"2024-04-18T22:27:18.725232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:18.725033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:18.725033+0000","last_clean_scrub_stamp":"2024-04-18T22:27:18.725033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:53:14.047183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"17'1","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:59.769787+0000","last_change":"2024-04-18T22:26:59.769850+0000","last_active":"2024-04-18T22:26:59.769787+0000","last_peered":"2024-04-18T22:26:59.769787+0000","last_clean":"2024-04-18T22:26:59.769787+0000","last_became_active":"2024-04-18T22:26:26.048797+0000","last_became_peered":"2024-04-18T22:26:26.048797+0000","last_unstale":"2024-04-18T22:26:59.769787+0000","last_undegraded":"2024-04-18T22:26:59.769787+0000","last_fullsized":"2024-04-18T22:26:59.769787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'1","last_scrub_stamp":"2024-04-18T22:26:59.769591+0000","last_deep_scrub":"17'1","last_deep_scrub_stamp":"2024-04-18T22:26:59.769591+0000","last_clean_scrub_stamp":"2024-04-18T22:26:59.769591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T06:24:05.029508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:51.800244+0000","last_change":"2024-04-18T22:26:51.800299+0000","last_active":"2024-04-18T22:26:51.800244+0000","last_peered":"2024-04-18T22:26:51.800244+0000","last_clean":"2024-04-18T22:26:51.800244+0000","last_became_active":"2024-04-18T22:26:37.250161+0000","last_became_peered":"2024-04-18T22:26:37.250161+0000","last_unstale":"2024-04-18T22:26:51.800244+0000","last_undegraded":"2024-04-18T22:26:51.800244+0000","last_fullsized":"2024-04-18T22:26:51.800244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:26:51.800067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:26:51.800067+0000","last_clean_scrub_stamp":"2024-04-18T22:26:51.800067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:26:53.488490+0000","scrub_duration":95,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"83'3","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:47.730120+0000","last_change":"2024-04-18T22:26:47.730181+0000","last_active":"2024-04-18T22:26:47.730120+0000","last_peered":"2024-04-18T22:26:47.730120+0000","last_clean":"2024-04-18T22:26:47.730120+0000","last_became_active":"2024-04-18T22:26:37.251010+0000","last_became_peered":"2024-04-18T22:26:37.251010+0000","last_unstale":"2024-04-18T22:26:47.730120+0000","last_undegraded":"2024-04-18T22:26:47.730120+0000","last_fullsized":"2024-04-18T22:26:47.730120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:47.729935+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:47.729935+0000","last_clean_scrub_stamp":"2024-04-18T22:26:47.729935+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:12:08.449282+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:14.639760+0000","last_change":"2024-04-18T22:27:14.639812+0000","last_active":"2024-04-18T22:27:14.639760+0000","last_peered":"2024-04-18T22:27:14.639760+0000","last_clean":"2024-04-18T22:27:14.639760+0000","last_became_active":"2024-04-18T22:26:37.239125+0000","last_became_peered":"2024-04-18T22:26:37.239125+0000","last_unstale":"2024-04-18T22:27:14.639760+0000","last_undegraded":"2024-04-18T22:27:14.639760+0000","last_fullsized":"2024-04-18T22:27:14.639760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:14.639562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:14.639562+0000","last_clean_scrub_stamp":"2024-04-18T22:27:14.639562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T09:45:27.979953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"83'3","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:54.842969+0000","last_change":"2024-04-18T22:26:54.843035+0000","last_active":"2024-04-18T22:26:54.842969+0000","last_peered":"2024-04-18T22:26:54.842969+0000","last_clean":"2024-04-18T22:26:54.842969+0000","last_became_active":"2024-04-18T22:26:26.053749+0000","last_became_peered":"2024-04-18T22:26:26.053749+0000","last_unstale":"2024-04-18T22:26:54.842969+0000","last_undegraded":"2024-04-18T22:26:54.842969+0000","last_fullsized":"2024-04-18T22:26:54.842969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-04-18T22:26:54.842768+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-04-18T22:26:54.842768+0000","last_clean_scrub_stamp":"2024-04-18T22:26:54.842768+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T22:31:45.020434+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:07.789190+0000","last_change":"2024-04-18T22:27:07.789246+0000","last_active":"2024-04-18T22:27:07.789190+0000","last_peered":"2024-04-18T22:27:07.789190+0000","last_clean":"2024-04-18T22:27:07.789190+0000","last_became_active":"2024-04-18T22:26:26.034353+0000","last_became_peered":"2024-04-18T22:26:26.034353+0000","last_unstale":"2024-04-18T22:27:07.789190+0000","last_undegraded":"2024-04-18T22:27:07.789190+0000","last_fullsized":"2024-04-18T22:27:07.789190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:07.788998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:07.788998+0000","last_clean_scrub_stamp":"2024-04-18T22:27:07.788998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T02:00:59.887879+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"17'1","reported_seq":162,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:26:53.082011+0000","last_change":"2024-04-18T22:26:53.082081+0000","last_active":"2024-04-18T22:26:53.082011+0000","last_peered":"2024-04-18T22:26:53.082011+0000","last_clean":"2024-04-18T22:26:53.082011+0000","last_became_active":"2024-04-18T22:26:26.068439+0000","last_became_peered":"2024-04-18T22:26:26.068439+0000","last_unstale":"2024-04-18T22:26:53.082011+0000","last_undegraded":"2024-04-18T22:26:53.082011+0000","last_fullsized":"2024-04-18T22:26:53.082011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'1","last_scrub_stamp":"2024-04-18T22:26:53.081817+0000","last_deep_scrub":"17'1","last_deep_scrub_stamp":"2024-04-18T22:26:53.081817+0000","last_clean_scrub_stamp":"2024-04-18T22:26:53.081817+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T10:15:27.621584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:08.715300+0000","last_change":"2024-04-18T22:27:08.715300+0000","last_active":"2024-04-18T22:27:08.715300+0000","last_peered":"2024-04-18T22:27:08.715300+0000","last_clean":"2024-04-18T22:27:08.715300+0000","last_became_active":"2024-04-18T22:26:37.241442+0000","last_became_peered":"2024-04-18T22:26:37.241442+0000","last_unstale":"2024-04-18T22:27:08.715300+0000","last_undegraded":"2024-04-18T22:27:08.715300+0000","last_fullsized":"2024-04-18T22:27:08.715300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T22:27:08.714964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T22:27:08.714964+0000","last_clean_scrub_stamp":"2024-04-18T22:27:08.714964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-19T23:13:47.467236+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-18T22:27:19.271897+0000","last_change":"2024-04-18T22:27:19.272106+0000","last_active":"2024-04-18T22:27:19.271897+0000","last_peered":"2024-04-18T22:27:19.271897+0000","last_clean":"2024-04-18T22:27:19.271897+0000","last_became_active":"2024-04-18T22:26:34.224296+0000","last_became_peered":"2024-04-18T22:26:34.224296+0000","last_unstale":"2024-04-18T22:27:19.271897+0000","last_undegraded":"2024-04-18T22:27:19.271897+0000","last_fullsized":"2024-04-18T22:27:19.271897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-04-18T22:27:19.271653+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-04-18T22:27:19.271653+0000","last_clean_scrub_stamp":"2024-04-18T22:27:19.271653+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T04:59:47.411167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":13734,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738434,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293160,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94078680,"statfs":{"total":96636764160,"available":96336568320,"internally_reserved":0,"allocated":1015808,"data_stored":974894,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":75,"seq":322122547214,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41888,"kb_used_data":984,"kb_used_omap":5,"kb_used_meta":40890,"kb_avail":94329952,"statfs":{"total":96636764160,"available":96593870848,"internally_reserved":0,"allocated":1007616,"data_stored":974278,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":6012,"internal_metadata":41871492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334744,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":94037096,"statfs":{"total":96636764160,"available":96293986304,"internally_reserved":0,"allocated":933888,"data_stored":382732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6867,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4417,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T22:27:28.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-18T22:27:28.388 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-18T22:27:28.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:27:28.710 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:27:28.710 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-18T22:22:31.467948+0000","modified":"2024-04-18T22:22:33.473846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-04-18T22:27:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-18T22:27:28.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-18T22:27:29.064 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:27:29.064 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-18T22:22:31.467948+0000","modified":"2024-04-18T22:22:33.473846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6803/2739131151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2739131151},{"type":"v1","addr":"172.21.15.32:6803","nonce":2739131151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-04-18T22:27:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-04-18T22:27:29.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T22:27:29.378 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-18T22:27:29.378 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"a1b9e9bc-8f1f-4426-9796-fae392070eec","created":"2024-04-18T22:22:10.044535+0000","modified":"2024-04-18T22:26:38.231275+0000","last_up_change":"2024-04-18T22:26:25.018371+0000","last_in_change":"2024-04-18T22:22:11.681436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T22:22:20.901399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T22:22:26.016101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-04-18T22:22:29.965363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-04-18T22:22:30.463150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57238913-8d4f-4ea8-a1a5-92ce7aceaf4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6801","nonce":4267325794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6803","nonce":4267325794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6807","nonce":4267325794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":4267325794},{"type":"v1","addr":"172.21.15.86:6805","nonce":4267325794}]},"public_addr":"172.21.15.86:6801/4267325794","cluster_addr":"172.21.15.86:6803/4267325794","heartbeat_back_addr":"172.21.15.86:6807/4267325794","heartbeat_front_addr":"172.21.15.86:6805/4267325794","state":["exists","up"]},{"osd":1,"uuid":"68bfe465-447e-4083-a8d5-e6f996b2bb2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":35,"up_from":75,"up_thru":78,"down_at":36,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6801","nonce":1652764673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6803","nonce":1652764673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6807","nonce":1652764673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1652764673},{"type":"v1","addr":"172.21.15.96:6805","nonce":1652764673}]},"public_addr":"172.21.15.96:6801/1652764673","cluster_addr":"172.21.15.96:6803/1652764673","heartbeat_back_addr":"172.21.15.96:6807/1652764673","heartbeat_front_addr":"172.21.15.96:6805/1652764673","state":["exists","up"]},{"osd":2,"uuid":"32ef3d14-8c6f-4e58-8026-6c0c96fbfe1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6801","nonce":2337964302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6803","nonce":2337964302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6807","nonce":2337964302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2337964302},{"type":"v1","addr":"172.21.15.196:6805","nonce":2337964302}]},"public_addr":"172.21.15.196:6801/2337964302","cluster_addr":"172.21.15.196:6803/2337964302","heartbeat_back_addr":"172.21.15.196:6807/2337964302","heartbeat_front_addr":"172.21.15.196:6805/2337964302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.941980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.992073+0000","dead_epoch":35},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T22:22:16.978727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T22:27:29.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-18T22:27:29.719 INFO:tasks.ceph.mds.a.smithi032.stderr: -1600> 2024-04-18T22:22:37.211+0000 7fc18edc0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-18T22:27:29.720 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-18T22:27:29.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-18T22:27:29.745 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-18T22:27:30.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-18T22:27:31.734 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-18T22:27:31.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-18T22:27:32.729 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-18T22:27:32.757 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-18T22:27:32.757 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-18T22:27:32.757 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:32.770 INFO:tasks.ceph.mds.a:Stopped 2024-04-18T22:27:32.770 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-18T22:27:32.770 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-18T22:27:32.770 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:34.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-18T22:27:38.873 INFO:tasks.ceph.osd.0:Stopped 2024-04-18T22:27:38.873 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-18T22:27:38.873 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:38.929 INFO:tasks.ceph.osd.1:Stopped 2024-04-18T22:27:38.930 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-18T22:27:38.930 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:39.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-18T22:27:39.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-18T22:27:45.031 INFO:tasks.ceph.osd.2:Stopped 2024-04-18T22:27:45.032 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-18T22:27:45.032 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-18T22:27:45.032 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:45.060 INFO:tasks.ceph.mgr.x:Stopped 2024-04-18T22:27:45.060 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-18T22:27:45.060 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:45.131 INFO:tasks.ceph.mgr.y:Stopped 2024-04-18T22:27:45.131 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-18T22:27:45.131 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-18T22:27:45.131 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:45.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~6s 2024-04-18T22:27:45.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-18T22:27:45.186 INFO:tasks.ceph.mon.a:Stopped 2024-04-18T22:27:45.186 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-18T22:27:45.186 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:45.247 INFO:tasks.ceph.mon.b:Stopped 2024-04-18T22:27:45.247 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-18T22:27:45.247 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T22:27:45.304 INFO:tasks.ceph.mon.c:Stopped 2024-04-18T22:27:45.304 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-18T22:27:45.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(MDS_TRIM\)' | egrep -v '\(MDS_SLOW_REQUEST\)' | egrep -v MDS_SLOW_METADATA_IO | egrep -v 'Behind on trimming' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-04-18T22:27:45.362 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-18T22:27:45.363 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-18T22:27:45.610 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:27:45.610 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-18T22:27:45.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-18T22:27:45.714 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-18T22:27:46.090 INFO:tasks.ceph:Archiving mon data... 2024-04-18T22:27:46.091 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/data/mon.a.tgz 2024-04-18T22:27:46.092 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-18T22:27:46.098 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.PAaWbMPygV 2024-04-18T22:27:46.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PAaWbMPygV 2024-04-18T22:27:46.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.PAaWbMPygV 2024-04-18T22:27:46.311 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.PAaWbMPygV is 402KB 2024-04-18T22:27:46.376 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.PAaWbMPygV 2024-04-18T22:27:46.381 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/data/mon.b.tgz 2024-04-18T22:27:46.381 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-18T22:27:46.430 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.6BWOrHMUFj 2024-04-18T22:27:46.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6BWOrHMUFj 2024-04-18T22:27:46.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.6BWOrHMUFj 2024-04-18T22:27:46.678 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.6BWOrHMUFj is 556KB 2024-04-18T22:27:46.741 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.6BWOrHMUFj 2024-04-18T22:27:46.746 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/data/mon.c.tgz 2024-04-18T22:27:46.746 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-18T22:27:46.794 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.MmdPE1t3k2 2024-04-18T22:27:46.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MmdPE1t3k2 2024-04-18T22:27:46.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.MmdPE1t3k2 2024-04-18T22:27:47.033 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.MmdPE1t3k2 is 556KB 2024-04-18T22:27:47.096 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.MmdPE1t3k2 2024-04-18T22:27:47.101 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-18T22:27:47.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T22:27:47.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T22:27:47.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T22:27:47.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T22:27:47.231 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T22:27:47.238 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T22:27:47.249 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T22:27:47.255 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T22:27:47.255 INFO:tasks.ceph:Archiving crash dumps... 2024-04-18T22:27:47.256 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi032/crash 2024-04-18T22:27:47.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T22:27:47.287 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi086/crash 2024-04-18T22:27:47.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T22:27:47.302 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi096/crash 2024-04-18T22:27:47.302 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T22:27:47.336 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi196/crash 2024-04-18T22:27:47.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T22:27:47.350 INFO:tasks.ceph:Compressing logs... 2024-04-18T22:27:47.350 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T22:27:47.353 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T22:27:47.355 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T22:27:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23009.log 2024-04-18T22:27:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21154.log 2024-04-18T22:27:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26728.log 2024-04-18T22:27:47.377 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23009.log.gz 2024-04-18T22:27:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21154.log.gz 2024-04-18T22:27:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21024.log 2024-04-18T22:27:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21102.log 2024-04-18T22:27:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26728.log.gz 2024-04-18T22:27:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21024.log.gz 2024-04-18T22:27:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21128.log 2024-04-18T22:27:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21561.log 2024-04-18T22:27:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21102.log: /var/log/ceph/ceph-client.admin.21128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21102.log.gz 2024-04-18T22:27:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25062.log 2024-04-18T22:27:47.380 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21128.log.gz 2024-04-18T22:27:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21561.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-18T22:27:47.380 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21561.log.gz 2024-04-18T22:27:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25336.log 2024-04-18T22:27:47.384 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25062.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24784.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26544.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25336.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24784.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23579.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5/var/log/ceph/ceph-client.admin.26544.log: --verbose -- /var/log/ceph/ceph-client.admin.25561.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26544.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25894.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23579.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21457.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25561.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25894.log.gz 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25811.log 2024-04-18T22:27:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23861.log 2024-04-18T22:27:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21457.log.gz 2024-04-18T22:27:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23916.log 2024-04-18T22:27:47.386 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25811.log.gz 2024-04-18T22:27:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22341.log 2024-04-18T22:27:47.386 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23861.log.gz 2024-04-18T22:27:47.387 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T22:27:47.398 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-18T22:27:47.398 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.739.log 2024-04-18T22:27:47.399 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.739.log.gz 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23885.log 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23916.log.gz 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26570.log 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22341.log.gz 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24426.log 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23885.log.gz 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21050.log 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26570.log.gz 2024-04-18T22:27:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25227.log 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24426.log.gz 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21050.log.gz 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22952.log 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23237.log 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25227.log.gz 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26284.log 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22952.log.gz 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23237.log.gz 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20922.log 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26213.log 2024-04-18T22:27:47.405 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26284.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25506.log 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20922.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26213.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26362.log 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24055.log 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25506.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21483.log 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26362.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24055.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22574.log 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22029.log 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21483.log.gz 2024-04-18T22:27:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22574.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22496.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22029.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.22029.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22869.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22522.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22496.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25422.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22869.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26388.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22522.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22003.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25422.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25282.log 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26388.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22003.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.22003.log.gz 2024-04-18T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.25946.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24645.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25282.log.gz 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23059.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25946.log.gz 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23920.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24645.log.gz 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21076.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23059.log.gz 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23553.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23920.log.gz 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26674.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21076.log.gz 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25778.log 2024-04-18T22:27:47.408 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23553.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21777.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26674.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26466.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25778.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26754.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21777.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25256.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26466.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21431.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26754.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23294.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25256.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24242.log 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21431.log.gz 2024-04-18T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23294.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23294.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25752.log 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24242.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24376.log 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21854.log 0.0% 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25752.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22445.log 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24376.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26055.log 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21854.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21405.log 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr: 48.1% -- replaced with /var/log/ceph/ceph-client.admin.22445.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26055.log.gz 2024-04-18T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23035.log 2024-04-18T22:27:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24079.log 2024-04-18T22:27:47.411 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21405.log.gz 2024-04-18T22:27:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23035.log.gz 2024-04-18T22:27:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21665.log 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24590.log 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24079.log.gz 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21665.log.gz 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23667.log 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26830.log 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24590.log.gz 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23667.log.gz 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21379.log 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21206.log 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26830.log.gz 2024-04-18T22:27:47.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21379.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21587.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21932.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21206.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21587.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22419.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26778.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21932.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22419.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24000.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25117.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26778.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24000.log.gz 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22055.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26804.log 2024-04-18T22:27:47.444 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25117.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22055.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24561.log 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.26804.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.23751.log 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26804.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24561.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25366.log 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26979.log 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23751.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25366.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23610.log 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22921.log 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26979.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23610.log.gz 2024-04-18T22:27:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22315.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22263.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22921.log.gz 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22315.log.gz 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22289.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.22800.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22263.log.gz 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22289.log.gz 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26926.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.22800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21234.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.22800.log.gz 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26926.log.gz 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22642.log 2024-04-18T22:27:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23149.log 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21234.log.gz 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22642.log.gz 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22107.log 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26953.log 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23149.log.gz 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22107.log.gz 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22778.log 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21262.log 2024-04-18T22:27:47.447 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26953.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22778.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24755.log 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24026.log 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21262.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24755.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23777.log 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22081.log 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24026.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23777.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24110.log 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25451.log 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22081.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24110.log.gz 2024-04-18T22:27:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22643.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24948.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25451.log.gz 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22643.log.gz 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25616.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21855.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24948.log.gz 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25616.log.gz 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20700.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21933.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21855.log.gz 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20700.log.gz 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22211.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26414.log 2024-04-18T22:27:47.449 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21933.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22211.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22641.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22747.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26414.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25700.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22641.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25724.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22747.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25186.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25700.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25724.log.gz 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23490.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21776.log 2024-04-18T22:27:47.450 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25186.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23490.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22977.log 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23382.log 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21776.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22977.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26214.log 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20842.log 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23382.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21323.log 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26214.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23636.log 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20842.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21323.log.gz 2024-04-18T22:27:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23832.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24700.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23636.log.gz 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23832.log.gz 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21613.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23206.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24700.log.gz 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21613.log.gz 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21698.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23206.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23206.log.gz 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21698.log.gz 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24165.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25477.log 2024-04-18T22:27:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26440.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24165.log.gz 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25477.log.gz 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20948.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24218.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26440.log.gz 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25395.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20948.log.gz 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24218.log.gz 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24923.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25395.log.gz 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24321.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26135.log 2024-04-18T22:27:47.453 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24923.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24808.log 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24321.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24533.log 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26135.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23092.log 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24808.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24291.log 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24533.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21925.log 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23092.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24291.log: 94.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23351.log 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24291.log.gz 2024-04-18T22:27:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26291.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21925.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23806.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23351.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22393.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26291.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23806.log: 93.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25645.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23806.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25033.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22393.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20305.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25645.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24729.log 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25033.log.gz 2024-04-18T22:27:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.20305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25142.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20305.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23123.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24729.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25142.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24483.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25920.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23123.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24483.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21349.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21535.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25920.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22838.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21349.log.gz 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26024.log 2024-04-18T22:27:47.456 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21535.log.gz 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24139.log 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22838.log.gz 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25671.log 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26024.log.gz 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24139.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24139.log.gz 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24894.log 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25671.log.gz 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26648.log 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24894.log.gz 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26136.log 2024-04-18T22:27:47.457 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24487.log 2024-04-18T22:27:47.465 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26648.log.gz 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26882.log 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26136.log.gz 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24487.log.gz 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20978.log 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26622.log 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26882.log.gz 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26058.log 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20978.log.gz 2024-04-18T22:27:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26622.log.gz 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25311.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24839.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26058.log.gz 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25311.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.25311.log.gz -- 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph.audit.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24839.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24839.log.gz 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22604.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21639.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22133.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22604.log.gz 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21639.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24868.log 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.21639.log.gz 2024-04-18T22:27:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26856.log 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22133.log.gz 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21769.log 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24868.log.gz 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21693.log 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26856.log: /var/log/ceph/ceph-client.admin.21769.log: gzip 95.1% -5 --verbose -- replaced with /var/log/ceph/ceph.audit.log.gz -- 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.23319.log 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26856.log.gz 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21769.log.gz 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21693.log.gz 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22237.log 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24453.log 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23319.log.gz 2024-04-18T22:27:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21699.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22237.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23433.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24453.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26492.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21699.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22159.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23433.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26492.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.26492.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22185.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26518.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22159.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26206.log 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22185.log.gz 2024-04-18T22:27:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26518.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21180.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26596.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26206.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22367.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21180.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26596.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23722.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26128.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22367.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23692.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23722.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26128.log.gz 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26700.log 2024-04-18T22:27:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23180.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23692.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24982.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26700.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23180.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.23180.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.24194.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26292.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24982.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23465.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24194.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26292.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.26292.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.21847.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24615.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23465.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21847.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21847.log.gz 2024-04-18T22:27:47.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24615.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25088.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25868.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.25088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20507.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25088.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25868.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25842.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25590.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20507.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23262.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25842.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25531.log 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25590.log.gz 2024-04-18T22:27:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24402.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23262.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25972.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25531.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26057.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24402.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25972.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20897.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.26057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24674.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26057.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20897.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.20897.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.23971.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25998.log 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24674.log.gz 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23971.log.gzgzip 2024-04-18T22:27:47.487 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.25172.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22548.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25998.log.gz 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.25172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25172.log.gz 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23407.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21509.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22548.log.gz 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23407.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23407.log.gz -5 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.24266.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24350.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21509.log.gz 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24266.log.gz 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24978.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23522.log 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24350.log.gz 2024-04-18T22:27:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.24978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24978.log.gz 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22895.log 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.23522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22721.log 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23522.log.gz 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22895.log.gz 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20723.log 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.22721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21297.log 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22721.log.gz 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.20723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20723.log.gz 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.21297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21297.log.gz 2024-04-18T22:27:47.489 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21651.log 2024-04-18T22:27:47.490 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-18T22:27:47.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.21651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21651.log.gz 2024-04-18T22:27:47.490 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21675.log 2024-04-18T22:27:47.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21699.log 2024-04-18T22:27:47.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.21675.log: /var/log/ceph/ceph-client.admin.21699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21675.log.gz 2024-04-18T22:27:47.490 INFO:teuthology.orchestra.run.smithi196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21699.log.gz 2024-04-18T22:27:47.563 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-18T22:27:47.604 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-18T22:27:47.683 INFO:teuthology.orchestra.run.smithi032.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-18T22:27:47.685 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-18T22:27:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m0.331s 2024-04-18T22:27:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m0.810s 2024-04-18T22:27:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.082s 2024-04-18T22:27:50.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-18T22:27:50.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-18T22:27:54.121 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-18T22:27:54.122 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-18T22:27:54.123 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m6.738s 2024-04-18T22:27:54.123 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m6.360s 2024-04-18T22:27:54.123 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.265s 2024-04-18T22:28:08.673 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-18T22:28:08.674 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-18T22:28:08.674 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m21.273s 2024-04-18T22:28:08.674 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m20.498s 2024-04-18T22:28:08.674 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.722s 2024-04-18T22:28:30.507 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.0.log: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-18T22:28:30.508 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-18T22:28:30.508 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m43.151s 2024-04-18T22:28:30.508 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m41.621s 2024-04-18T22:28:30.508 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m1.441s 2024-04-18T22:28:30.509 INFO:tasks.ceph:Archiving logs... 2024-04-18T22:28:30.509 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi032/log 2024-04-18T22:28:30.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T22:28:30.790 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi086/log 2024-04-18T22:28:30.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T22:28:36.561 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi096/log 2024-04-18T22:28:36.562 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T22:28:37.396 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi196/log 2024-04-18T22:28:37.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T22:28:40.162 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-18T22:28:40.172 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-18T22:28:40.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T22:28:40.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T22:28:40.177 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T22:28:40.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T22:28:40.299 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-18T22:28:40.308 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-18T22:28:40.309 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:28:40.311 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:28:40.314 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:28:40.316 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T22:28:40.322 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:28:40.322 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-04-18T22:28:40.322 INFO:teuthology.orchestra.run.smithi032.stdout:+hv01.front.sepi 67.205.162.81 3 u 51 64 377 0.110 -1.964 0.776 2024-04-18T22:28:40.322 INFO:teuthology.orchestra.run.smithi032.stdout:*hv02.front.sepi 63.231.80.2 3 u 49 64 377 0.068 -0.463 0.696 2024-04-18T22:28:40.323 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 74.6.168.72 3 u 41 64 377 0.065 -1.465 1.573 2024-04-18T22:28:40.323 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:28:40.324 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:28:40.324 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-04-18T22:28:40.324 INFO:teuthology.orchestra.run.smithi086.stdout:+hv01.front.sepi 67.205.162.81 3 u 47 64 377 0.101 -2.301 1.210 2024-04-18T22:28:40.325 INFO:teuthology.orchestra.run.smithi086.stdout:*hv02.front.sepi 63.231.80.2 3 u 51 64 377 0.069 +0.085 1.827 2024-04-18T22:28:40.325 INFO:teuthology.orchestra.run.smithi086.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 64 377 0.073 +0.730 1.835 2024-04-18T22:28:40.325 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:28:40.330 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:28:40.330 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-04-18T22:28:40.330 INFO:teuthology.orchestra.run.smithi196.stdout:+hv01.front.sepi 67.205.162.81 3 u 50 64 377 0.057 -2.315 0.960 2024-04-18T22:28:40.330 INFO:teuthology.orchestra.run.smithi196.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 377 0.061 -0.285 1.235 2024-04-18T22:28:40.330 INFO:teuthology.orchestra.run.smithi196.stdout:*hv03.front.sepi 74.6.168.72 3 u 51 64 377 0.091 +0.209 1.182 2024-04-18T22:28:40.331 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:28:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T22:28:40.336 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-04-18T22:28:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi 67.205.162.81 3 u 24 64 7 0.146 +24.985 1.385 2024-04-18T22:28:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi 63.231.80.2 3 u 22 64 7 0.135 +26.439 1.433 2024-04-18T22:28:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi 74.6.168.72 3 u 20 64 7 0.147 +27.011 1.400 2024-04-18T22:28:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T22:28:40.337 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-18T22:28:40.349 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-18T22:28:40.349 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-18T22:28:40.390 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-18T22:28:40.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-18T22:28:40.559 INFO:teuthology.task.internal:Duration was 1012.706546 seconds 2024-04-18T22:28:40.559 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-18T22:28:40.660 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-18T22:28:40.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T22:28:40.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T22:28:40.665 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T22:28:40.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T22:28:40.708 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-18T22:28:40.708 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-04-18T22:28:40.708 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T22:28:40.750 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-18T22:28:40.751 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T22:28:40.762 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-04-18T22:28:40.762 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T22:28:40.774 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-18T22:28:40.774 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T22:28:40.784 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-18T22:28:40.784 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T22:28:40.794 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T22:28:40.806 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T22:28:40.816 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T22:28:40.872 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-18T22:28:40.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T22:28:40.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T22:28:40.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T22:28:40.878 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T22:28:40.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-18T22:28:40.981 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-18T22:28:40.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T22:28:40.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T22:28:41.006 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T22:28:41.020 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T22:28:41.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-18T22:28:41.050 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.054 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.063 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-18T22:28:41.063 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-04-18T22:28:41.065 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.091 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-04-18T22:28:41.091 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-18T22:28:41.189 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:28:41.194 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:28:41.200 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:28:41.205 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T22:28:41.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:28:41.237 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-18T22:28:41.250 INFO:teuthology.task.internal:Transferring archived files... 2024-04-18T22:28:41.251 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi032 2024-04-18T22:28:41.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T22:28:41.274 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi086 2024-04-18T22:28:41.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T22:28:41.295 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi096 2024-04-18T22:28:41.296 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T22:28:41.314 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:32:03-powercycle-squid-distro-default-smithi/7662815/remote/smithi196 2024-04-18T22:28:41.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T22:28:41.336 INFO:teuthology.task.internal:Removing archive directory... 2024-04-18T22:28:41.336 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T22:28:41.339 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T22:28:41.341 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T22:28:41.357 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T22:28:41.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-18T22:28:41.393 INFO:teuthology.task.internal:Not uploading archives. 2024-04-18T22:28:41.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-18T22:28:41.483 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-18T22:28:41.484 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T22:28:41.486 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T22:28:41.489 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T22:28:41.490 INFO:teuthology.orchestra.run.smithi032.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 22:28 /home/ubuntu/cephtest 2024-04-18T22:28:41.491 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T22:28:41.493 INFO:teuthology.orchestra.run.smithi086.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 22:28 /home/ubuntu/cephtest 2024-04-18T22:28:41.495 INFO:teuthology.orchestra.run.smithi096.stdout: 658463 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 22:28 /home/ubuntu/cephtest 2024-04-18T22:28:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: 658463 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 22:28 /home/ubuntu/cephtest 2024-04-18T22:28:41.499 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-18T22:28:41.547 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-18T22:28:41.751 INFO:teuthology.run:Summary data: description: powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/ubuntu_latest tasks/admin_socket_objecter_requests thrashosds-health} duration: 1012.7065463066101 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-18T22:28:41.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T22:28:41.824 INFO:teuthology.run:pass